AIRCC's International Journal of Computer Science and Information Technology

Open Access Open Access  Restricted Access Subscription Access

Steam++ An Extensible End-to-end Framework For Developing Iot Data Processing Applications In The Fog


Affiliations
1 University of Vale do Rio dos Sinos - UNISINOS, RS, Brazil
2 Pontifical Catholic University of Rio Grande do Sul, RS, Brazil
 

IoT applications usually rely on cloud computing services to perform data analysis such as filtering, aggregation, classification, pattern detection, and prediction. When applied to specific domains, the IoT needs to deal with unique constraints. Besides the hostile environment such as vibration and electricmagnetic interference, resulting in malfunction, noise, and data loss, industrial plants often have Internet access restricted or unavailable, forcing us to design stand-alone fog and edge computing solutions. In this context, we present STEAM++, a lightweight and extensible framework for real-time data stream processing and decision-making in the network edge, targeting hardware-limited devices, besides proposing a micro-benchmark methodology for assessing embedded IoT applications. In real-case experiments in a semiconductor industry, we processed an entire data flow, from values sensing, processing and analysing data, detecting relevant events, and finally, publishing results to a dashboard. On average, the application consumed less than 500kb RAM and 1.0% of CPU usage, processing up to 239 data packets per second and reducing the output data size to 14% of the input raw data size when notifying events.

Keywords

Edge Computing, IoT, Fog, Stream Processing, Data Analysis, Framework.
User
Notifications
Font Size

  • A. Protopsaltis, P. Sarigiannidis, D. Margounakis, and A. Lytos, “Data visualization in internet of things: Tools, methodologies, and challenges,” in Proceedings of the 15th International Conference on Availability, Reliability and Security, ARES ’20, pp. 1–11, Association for Computing Machinery, 2020.

  • J. He, J. Rong, L. Sun, H. Wang, Y. Zhang, and J. Ma, “A framework for cardiac arrhythmia detection from iot-based ecgs,” World Wide Web, vol. 23, no. 5, pp. 2835–2850, 2020.

  • I. Figueiredo, P. Esteves, and P. Cabrita, “Water wise–a digital water solution for smart cities and water management entities,” Procedia Computer Science, vol. 181, pp. 897–904, 2021.

  • M. Mohammad Yousef, “Big Data Analytics in Health Care: A Review Paper,” International Journal of Computer Science and Information Technology, vol. 13, pp. 17–28, apr 2021.

  • S. Ganesh and R. Gurumoorthy, “Real Time Vigilance Detection using Frontal EEG,” International Journal of Computer Science and Information Technology, vol. 13, pp. 49–56, feb 2021.

  • A. S. Albahri, J. K. Alwan, Z. K. Taha, S. F. Ismail, R. A. Hamid, A. Zaidan, O. S. Albahri, B. Zaidan, A. Alamoodi, and M. Alsalem, “Iot-based telemedicine for disease prevention and health promotion: State-of-the-art,” Journal of Network and Computer Applications, vol. 173, p. 102873, 2021.

  • R. Akhter and S. A. Sofi, “Precision agriculture using iot data analytics and machine learning,” Journal of King Saud University - Computer and Information Sciences, 2021.

  • Y. Liu, T. Dillon, W. Yu, W. Rahayu, and F. Mostafa, “Noise removal in the presence of significant anomalies for industrial iot sensor data in manufacturing,” IEEE Internet of Things Journal, vol. 7, no. 8, pp. 7084– 7096, 2020.

  • A. Gaddam, T. Wilkin, M. Angelova, and J. Gaddam, “Detecting sensor faults, anomalies and outliers in the internet of things: A survey on the challenges and solutions,” Electronics, vol. 9, no. 3, p. 511, 2020.

  • B. Chander and G. Kumaravelan, “Outlier detection strategies for wsns: A survey,” Journal of King Saud University - Computer and Information Sciences, 2021.

  • S. Munirathinam, “Drift detection analytics for iot sensors,” Procedia Computer Science, vol. 180, pp. 903–912, 2021.

  • G. Ortiz, M. Zouai, O. Kazar, A. Garcia de Prado, and J. Boubeta Puig, “Atmosphere: Context and situational-aware collaborative iot architecture for edge-fog-cloud computing,” Computer Standards & Interfaces, p. 103550, 2021.

  • D. Corral Plaza, G. Ortiz, I. Medina Bulo, and J. Boubeta Puig, “Medit4cep-sp: A model-driven solution to improve decision-making through user-friendly management and real-time processing of heterogeneous data streams,” Knowledge-Based Systems, vol. 213, p. 106682, 2021.

  • C. Liu, X. Su, and C. Li, “Edge computing for data anomaly detection of multi-sensors in underground mining,” Electronics, vol. 10, no. 3, p. 302, 2021.

  • A. Fertier, G. Martin, A.M. Barthe Delano, J. Lesbegueries, A. Montarnal, S. Truptil, F. Benaben, and N. Salatge, “Managing events to improve situation awareness and resilience in a supply chain,” Computers in Industry, vol. 132, p. 103488, 2021.

  • M. Al Rakhami, A. Gumaei, M. Alsahli, M. M. Hassan, A. Alamri, A. Guerrieri, and G. Fortino, “A lightweight and cost effective edge intelligence architecture based on containerization technology,” World Wide Web, vol. 23, no. 2, pp. 1341–1360, 2020.

  • M. Iorga, L. Feldman, R. Barton, M. J. Martin, N. Goren, and C. Mahmoudi, “Fog computing conceptual model,” tech. rep., National Institute of Standards and Technology, Gaithersburg, MD, mar 2018.

  • F. De Vita, D. Bruneo, and S. K. Das, “A novel data collection framework for telemetry and anomaly detection in industrial iot systems,” in 2020 IEEE/ACM Fifth International Conference on Internet of Things Design and Implementation (IoTDI), pp. 245–251, IEEE, 2020.

  • C. Bourelly, A. Bria, L. Ferrigno, L. Gerevini, C. Marrocco, M. Molinara, G. Cerro, M. Cicalini, and A. Ria, “A preliminary solution for anomaly detection in water quality monitoring,” in 2020 IEEE International Conference on Smart Computing (SMARTCOMP), pp. 410–415, IEEE, 2020.

  • C. Yin, B. Li, and Z. Yin, “A distributed sensing data anomaly detection scheme,” Computers & Security, vol. 97, p. 101960, 2020.

  • S. K. YR and H. Champa, “Iot streaming data outlier detection and sensor data aggregation,” in 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC), pp. 150– 155, IEEE, 2020

  • L. Greco, P. Ritrovato, and F. Xhafa, “An edge-stream computing infrastructure for real-time analysis of wearable sensors data,” Future Generation Computer Systems, vol. 93, pp. 515–528, 2019.

  • M. M. Gomes, R. da Rosa Righi, C. A. da Costa, and D. Griebler, “Simplifying iot data stream enrichment and analytics in the edge,” Computers & Electrical Engineering, vol. 92, p. 107110, 2021.

  • Q. P. He and J. Wang, “Statistical process monitoring as a big data analytics tool for smart manufacturing,” Journal of Process Control, vol. 67, pp. 35–43, 2018.


Abstract Views: 173

PDF Views: 94




  • Steam++ An Extensible End-to-end Framework For Developing Iot Data Processing Applications In The Fog

Abstract Views: 173  |  PDF Views: 94

Authors

Márcio Miguel Gomes
University of Vale do Rio dos Sinos - UNISINOS, RS, Brazil
Rodrigo da Rosa Righi
University of Vale do Rio dos Sinos - UNISINOS, RS, Brazil
Cristiano André da Costa
University of Vale do Rio dos Sinos - UNISINOS, RS, Brazil
Dalvan Griebler
Pontifical Catholic University of Rio Grande do Sul, RS, Brazil

Abstract


IoT applications usually rely on cloud computing services to perform data analysis such as filtering, aggregation, classification, pattern detection, and prediction. When applied to specific domains, the IoT needs to deal with unique constraints. Besides the hostile environment such as vibration and electricmagnetic interference, resulting in malfunction, noise, and data loss, industrial plants often have Internet access restricted or unavailable, forcing us to design stand-alone fog and edge computing solutions. In this context, we present STEAM++, a lightweight and extensible framework for real-time data stream processing and decision-making in the network edge, targeting hardware-limited devices, besides proposing a micro-benchmark methodology for assessing embedded IoT applications. In real-case experiments in a semiconductor industry, we processed an entire data flow, from values sensing, processing and analysing data, detecting relevant events, and finally, publishing results to a dashboard. On average, the application consumed less than 500kb RAM and 1.0% of CPU usage, processing up to 239 data packets per second and reducing the output data size to 14% of the input raw data size when notifying events.

Keywords


Edge Computing, IoT, Fog, Stream Processing, Data Analysis, Framework.

References