Author Details

Big data is the most important trend that is defining the new emerging analytical tools. Big data has various applications in different areas like traffic control, weather forecasting, fraud detection, security, education and health care. Extraction of knowledge from massive amount of data sets has become a challenging task. Big data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, store and analyze it within a tolerable elapsed time. Due to widespread usage of many computing devices such as smart phones, laptops, wearable computing devices; the data processing over the internet has exceeded more than the modern computers can handle. Due to this high growth rate, the term Big Data is envisaged. However, the fast growth rate of such large data generates numerous challenges, such as data inconsistency and incompleteness, scalability, timeliness, and security. The question that arises now is how to develop a high performance platform to efficiently analyze big data and how to design an appropriate mining algorithm to find the useful things from big data. This paper begins with a brief introduction to the big data technology and its importance and also focuses on various challenges and issues that need to be emphasized. The tools used in big data technology are also discussed in detail.

Keywords

Big Data, Hadoop, Map Reduce, Pig, Hive, Hbase.

Full Text

References

https://www.idc.com/prodserv/4Pillars/bigdata

A, Katal, Wazid M, and Goudar R.H. "Big data: Issues, challenges, tools and Good practices." Noida: 2013, pp. 404 – 409, 8-10 Aug. 2013.]

Almeida, F., and Calistru, C, "The Main Challenges and Issues of Big Data Management", International Journal of Research Studies in Computing, 2(1), 2013, pp. 11-20.

Apache Hadoop (2013). HDFS Architecture Guide [Online]. Available: https://hadoop. apache.org/docs/r1.2.1/hdfs_design.ht

Amrit pal, Pinki Aggrawal, Kunal Jain, Sanjay Aggrawal “A Performance Analysis of MapReduce Task with Large Number of Files Dataset in Big Data using Hadoop” Forth International Conference on Communication Systems and Network Technologies, 2014.

Rahm, E., & Hai Do, H. (2000). Data cleaning: problems and current approaches. Bulletin of the Technical Committee on Data Engineering, 23(4), 3-13.),

Apache Hadoop (2013). HDFS Architecture Guide [Online]. Available: https://hadoop.apache.org/docs/r1.2.1/hdfs_design.ht

Intel, “Big Data Analaytics,”2012, http://www.intel.com/content/dam/www/public/us/en/documents/reports/data-insightspeer-research-report.pdf

Survey on Data Compression in Big Data by Using Various Methods

Abstract Views :201 | PDF Views:4

Authors

N. Kavitha ¹, K. Subhadra ²

Affiliations
1 Department of Computer Science, Nehru Arts and Science College, Coimbatore, IN
2 PG & Research Department Computer Science, Nehru Arts and Science College, Coimbatore, IN

Source

Digital Signal Processing, Vol 10, No 1 (2018), Pagination: 10-11

Abstract

Now a days Big sensing data is prevalent in both industry and scientific research applications where the data is generated with high volume and velocity. Cloud computing act as a stack of massive, storage, and software services in a scalable manner. Current big sensing data uses cloud computing technique for this purpose. Here we are using data compression techniques. Based on specific on cloud data compression requirements we propose a scalable data compression approach based on similarity among chunks. Map Reduce algorithm is used for this purpose. Aprominent parallel data processing tool is gaining significant momentum from both industry and academia as the volume of data to analyze grows rapidly. This survey intends to assist the data base and open source communities in understanding various technical aspects of the MAPREDUCE framework. As the name suggests reducer phase take place in two sections. Map Reduce is used as an algorithm for the implementation to achieve extra scalability on Cloud. In general, big data is a collection of data sets so large and complex that it becomes extremely difficult to process with on-hand database management systems or traditional data processing tools. It represents the progress of the human cognitive processes, usually includes data sets with sizes beyond the ability of current technology, method and theory to capture, manage and process the data within a tolerable elapsed time .The big sensing data from different kinds of sensing systems is high heterogeneous, and it has typical characteristics of common real world big data. They are five ‘V’s, Volume, Variety, Velocity, Veracity and Veracity. Data chunk similarity can significantly improve data compression efficiency with affordable data accuracy loss. But due to the size and speed of big sensing data in real world, the current data compression and reduction techniques still need to be improved It has been well recognized that big sensing data or big data sets from mesh networks such as sensor systems and social networks can take the form of big graph data. To process those big graph data, current techniques normally introduce complex and multiple iterations. In order to cope with that huge volume big sensing data, different techniques can have been developed on-line or off-line, centralized or distributed.

Full Text

References

“Big data: science in petabyte era: community cleverness required” Nature 455 (7209): 1, 2008.

S. Tsuchiya, Y. Sakamoto, Y Tsuchimoto and V. Lee, “Big Data Processing in cloud environments” FUJITSU Science and Technology Journal, 48(2) 159-168 2002.

K.H. Lee, Y. J. Lee, H. Choi, Y.D. Chung and B. Moon, “Parellel data processing with Mapreduce: a survey” ACM SIGMOD record 40(4): 11-20, 2012.

Big Data beyond Mapreduce: Google’s Big Data Papers, http://architects.dzone.com/articles/big-data-beyond-mapreduce, accessed on November 20 2015.

Internet of Things (IoT) with New Prespectives

Abstract Views :192 | PDF Views:4

Authors

L. M. Mary Jelba ¹, N. Kavitha ¹

Affiliations
1 PG & Research Department of Computer Science, Nehru Arts and Science College, Coimbatore, IN

Source

Digital Signal Processing, Vol 10, No 1 (2018), Pagination: 12-13

Abstract

Internet of Things (IoT) is connected devices with interactions ie. Any service which allowed to interact with others. For example: any person, any device, any service, any place or any path can connect with IoT. IoT is used in various domains such as smart transport, health, business, etc,... and finally connected together in network.

Keywords

IoT Architecture, Technologies, IoT Cloud, Future of IoT.

Full Text

References

Z. Cai, Z. He, X. Guan, and Y. Li. Collective data-sanitization for preventing sensitive information inference attacks in social networks. IEEE Transactions on Dependable and Secure Computing, 2017.

F. Chen, T. Xiang, X. Fu, and W. Yu. User differentiated verifiable file search on the cloud. IEEE Transactions on Services Computing, 2017.

Z. Chen, G. Xu, V. Mahalingam, L. Ge, J. Nguyen, W. Yu, and C. Lu. A cloud computing based network monitoring and threat detection system for critical infrastructures. Big Data Research, 3:10–23, 2016.

W. Gao, J. Nguyen, W. Yu, C. Lu, and D. Ku. Assessing performance of constrained application protocol (coap) in manet using emulation. In Proc. of ACM International Conference on Reliable & Convergent Systems (RACS), 2016.

G. Kalnoor and J. Agarkhed. Qos based multipath routing for intrusion detection of sinkhole attack in wireless sensor networks. In Proc. of 2016 International Conference on Circuit, Power and Computing Technologies (ICCPCT), March 2016.

S. Mallapuram, N. Ngwum, F. Yuan, C. Lu, and W. Yu. Smart city: The state of the art, datasets, and evaluation platforms. In Proc. of 16th IEEE/ACIS International Conference on Computer and Information Science (ICIS), 2017.

K. Sha, W. Wei, A. Yang, and W. Shi. Security in internet of things: Opportunities and challenges. In Proceedings of the International Conference on Identification, Information & Knowledge in the Internet of Things, October 2016.

L. Zhang, Z. Cai, and X. Wang. Fakemask: A novel privacy preserving approach for smartphones. IEEE Transactions on Network and Service Management, 13(2):335–348, 2016.

W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu. Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5):637–646, October 2016.

A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, “Internet of Things: A survey on enabling technologies, protocols, and applications,” IEEE Commun. Surv. Tutor., vol. 17, no. 4, pp. 2347–2376, 2015.

N. Zhang, N. Cheng, A. T. Gamage, K. Zhang, J. W. Mark, and X. Shen, “Cloud assisted HetNets toward 5G wireless networks,” IEEE Commun. Mag., vol. 53, no. 6, pp. 59–65, 2015.

M. Natkaniec, K. Kosek-Szott, S. Szott, and G. Bianchi, “A survey of medium access mechanisms for providing QoS in ad-hoc networks,” IEEE Commun. Surv. Tutor., vol. 15, no. 2, pp. 592–620, 2013.

A. Aijaz and A. H. Aghvami, “Cognitive machine-to-machine commu-nications for Internet-of-Things: A protocol stack perspective,” IEEE Internet Things J., vol. 2, no. 2, pp. 103–112, 2015.

A. Rajandekar and B. Sikdar, “A survey of MAC layer issues and pro-tocols for machine-to-machine communications,” IEEE Internet Things J., vol. 2, no. 2, pp. 175–186, 2015.

J. Ren, Y. Zhang, N. Zhang, D. Zhang, and X. Shen, “Dynamic channel access to improve energy efficiency in cognitive radio sensor networks,” IEEE Trans. Wireless Commun., vol. 15, no. 5, pp. 3143–3156, 2016.

D. Zhang, Z. Chen, J. Ren, N. Zhang, M. Awad, H. Zhou, and X. Shen, “Energy harvesting-aided spectrum sensing and data transmission in het-erogeneous cognitive radio sensor network,” IEEE Trans. Veh. Technol., vol. 66, no. 1, pp. 831–843, 2017.

Q. Ye and W. Zhuang, “Distributed and adaptive medium access control for Internet-of-Things-enabled mobile networks,” IEEE Internet Things J., to appear. DOI: 10.1109/JIOT.2016.2566659.

H. Ma, L. Liu, A. Zhou, and D. Zhao, “On networking of internet of things: Explorations and challenges,” IEEE Internet of Things Journal, vol. 3, no. 4, pp. 441–452, Aug 2016.

E. C.-H. Ngai, M. B. Srivastava, and J. Liu, “Context-aware sensor data dissemination for mobile users in remote areas,” in Proc. of INFOCOM, 2012, pp. 2711–2715.

L. Cheng, J. Niu, M. D. Francesco, S. K. Das, C. Luo, and Y. Gu, “Seamless streaming data delivery in cluster-based wireless sensor networks with mobile elements,” IEEE Systems Journal, vol. 10, no. 2, pp. 805–816, June 2016.

A. Mehrabi and K. Kim, “Maximizing data collection throughput on a path in energy harvesting sensor networks using a mobile sink,” IEEE Transactions on Mobile Computing, vol. 15, no. 3, pp. 690–704, March 2016.

S. Yang, U. Adeel, Y. Tahir, and J. A. McCann, “Practical opportunistic data collection in wireless sensor networks with mobile sinks,” IEEE Transactions on Mobile Computing, vol. 16, no. 5, pp. 1420–1433, May 2017.

M. Vucinic, B. Tourancheau, F. Rousseau, A. Duda, L. Damon, and R.Guizzetti, “OSCAR: Object security architecture for the Internet of Things”, Ad Hoc Networks, 32, pp.3-16, 2015.

S. Cirani, M. Picone, P. Gonizzi, L. Veltri, and G. Ferrari, “IoT-OAS: an oauth-based authorization service architecture for secure services in IoT scenarios”, IEEE Sensors Journal, vol. 15, no. 2, pp.1224-1234, 2015.

Username
Password
Remember me