International Journal of Advanced Networking and Applications

Open Access Open Access  Restricted Access Subscription Access

Prototype Intelligent Log-based Intrusion Detection System


Affiliations
1 School of Informatics and Innovative Systems, Jaramogi Oginga Odinga University of Science and Technology, Bondo, Kenya
 

The maintenance of web server security is a daunting task today. Threats arise from hardware failures, software flaws, tentative probing and worst of all malicious attacks. Analysing server logs to detect suspicious activities is regarded as a key form of defence, however, their sheer size makes human log analysis challenging. Additionally, traditional intrusion detection systems rely on methods based on pattern-matching techniques which are not sustainable given the high rates at which new attack techniques are launched every day. The aim of this paper is to develop a proto-type intelligent log based intrusion detection system that can detect known and unknown intrusions automatically. Under a data mining framework, the intrusion detection system is trained with unsupervised learning algorithms specifically the k-means algorithm and the One Class SVM (Support Vector Machine) algorithm. The development of the prototype system is limited to machine generated logs due to lack of real access log files. However, the system’s development and implementation proved to be up to 85% accurate in detecting anomalous log patterns within the test logs.

Keywords

Prototype, Intrusion Detection, Log-Based, Data Mining
User
Notifications
Font Size

  • Amoli, P. V., Hamalainen, T., David, G., Zolotukhin, M., & Mirzamohammad, M. (2016). Unsupervised Network Intrusion Detection Systems for Zero-Day Fast-Spreading Attacks and Botnets. JDCTA (International Journal of Digital Content Technology and its Applications, Volume 10 Issue 2, 1-13.

  • CERT Coordination Center (CERT/CC). CERT/CC Statistics 1988-2003. http://www.cert.org/stats/cert_stats.html#incidents

  • CISCO Systems Ltd White paper:. The Science of Intrusion Detection System Attack Identification . http://www.cisco.com/en/US/products/sw/securs w/ps2113/products_whitepaper09186a008009233 4.shtml. last accessed December 2016 last accessed December 2016

  • Coates, A., Lee, H., & Ng, A. Y. (2010). An analysis of single-layer networks in unsupervised feature learning. Ann Arbor, 1001(48109), 2.

  • Deepa H. Kulkarni Computational Statistics and Predictive Analysis in Machine Learning. (2016). International Journal Of Science And Research (IJSR), 5(1), 1521-1524. http://dx.doi.org/10.21275/v5i1.nov152818 last accessed February 2017

  • Forrest, S., Perelson, A. S., Allen, L., & Cherukuri, R. (1994, May). Self-nonself discrimination in a computer. In Research in Security and Privacy, 1994. Proceedings., 1994 IEEE Computer Society Symposium on (pp. 202212). IEEE.

  • Gardner, A. B., Krieger, A. M., Vachtsevanos, G., & Litt, B. (2006). One-class novelty detection for seizure analysis from intracranial EEG. Journal of Machine Learning Research, 7(Jun), 1025-1044.

  • Gitau, J. M. (2016) Automated Log Analysis Using AI: Intelligent Intrusion Detection System. Jaramogi Odinga Oginga University of Science andTechnology. http://jooust.ac.ke/projects/siis/2016/JGM-102016.pdf last accessed February 2017 9. Github https://github.com/kiritbasu/FakeApache-Log-Generator

  • Hand, D. J., Mannila, H., & Smyth, P. (2001). Principles of data mining. MIT press.

  • Hinton, G. E., & Sejnowski, T. J. (1999). Unsupervised learning: foundations of neural computation. MIT press

  • Kanungo, T, Mount, D. M., Netanyahu, N. S., Piatko, C. D., Silverman, R. and Wu, A. Y. 2002. An efficient k-means clustering algorithm: Analysis and implementation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7):881–892.

  • Li, K. L., Huang, H. K., Tian, S. F., & Xu, W. (2003, November). Improving one-class SVM for anomaly detection. Machine Learning and Cybernetics, 2003 International Conference Vol. 5, pp. 3077-3081. IEEE.

  • Li, W. (2013). Automatic Log Analysis using Machine Learning: Awesome Automatic Log Analysis version 2.0. http://uu.divaportal.org/smash/get/diva2:667650/FULLTEXT01.pdf last accessed December 2016

  • Ma, P. (2003). Log Analysis-Based Intrusion Detection via Unsupervised Learning. Master of Science, School of Informatics, University of Edinburgh.

  • Manevitz, L. M., & Yousef, M. (2001). One-class SVMs for document classification. Journal of Machine Learning Research, 2(Dec), 139-154.

  • Markou, M., & Singh, S. (2003). Novelty detection: a review—part 1: statistical approaches. Signal processing, 83(12), 24812497.

  • Matherson, K. (2015). Machine Learning Log File Analysis. http://docplayer.net/10128120Machine-learning-log-file-analysis.html

  • Muller, K. R., Mika, S., Ratsch, G., Tsuda, K., & Scholkopf, B. (2001). An introduction to kernelbased learning algorithms. IEEE transactions on neural networks, 12(2), 181-201.

  • Parzen, E. (1962). On estimation of a probability density function and mode. The annals of mathematical statistics, 33(3), 1065-1076.

  • Patil , A. S and Patil, D. R. Post-Attack Intrusion Detection using Log Files Analysis. International Journal of Computer Applications 127(18):19-21, October 2015. Foundation of Computer Science (FCS), NY, USA.. http://dx.doi.org/10.5120/ijca2015906731 last accessed December 2016

  • Ryan, J., Lin, M. J., & Miikkulainen, R. (1998). Intrusion detection with neural networks. Advances in neural information processing systems, 943-949.

  • Schultz, M. G., Eskin, E., Zadok, E., Bhattacharyya, M., & Stolfo, S. J. (2001, June). MEF: Malicious Email Filter-A UNIX Mail Filter That Detects Malicious Windows Executables. In USENIX Annual Technical Conference, FREENIX Track (pp. 245-252).

  • Scikit-learn: machine learning in Python — scikit-learn 0.18.1 documentation. (2016). Scikitlearn.org. Retrieved 2 December 2016, from http://scikit-learn.org/stable/

  • Svensson, C. (2015). Automatic Log Analysis System Integration: Message Bus Integration in a Machine Learning Environment. http://www.divaportal.org/smash/get/diva2:81853 8/FULLTEXT01.pdf last accessed February 2017

  • Yen, T. F., Oprea, A., Onarlioglu, K., Leetham, T., Robertson, W., Juels, A., & Kirda, E. (2013, December). Beehive: Large-scale log analysis for detecting suspicious activity in enterprise networks. In Proceedings of the 29th Annual Computer Security Applications Conference (pp. 199-208). ACM.

  • Zwietasch, T. (2014). Detecting anomalies in system log files using machine learning techniques . http://dx.doi.org/10.18419/opus3454 last accessed February 2017

  • Rai, K., Davi, M. S. & Guleria, A. (2016), Decision Tree Based Algorithm for Intrusion Detection: Int. J. Advanced Networking and Applications, Volume: 07 Issue: 04 Pages: 28282834 (2016) ISSN: 0975-0290.


Abstract Views: 209

PDF Views: 1




  • Prototype Intelligent Log-based Intrusion Detection System

Abstract Views: 209  |  PDF Views: 1

Authors

M. Gitau Joseph
School of Informatics and Innovative Systems, Jaramogi Oginga Odinga University of Science and Technology, Bondo, Kenya
J. Rodrigues Anthony
School of Informatics and Innovative Systems, Jaramogi Oginga Odinga University of Science and Technology, Bondo, Kenya
Abuonji Paul
School of Informatics and Innovative Systems, Jaramogi Oginga Odinga University of Science and Technology, Bondo, Kenya

Abstract


The maintenance of web server security is a daunting task today. Threats arise from hardware failures, software flaws, tentative probing and worst of all malicious attacks. Analysing server logs to detect suspicious activities is regarded as a key form of defence, however, their sheer size makes human log analysis challenging. Additionally, traditional intrusion detection systems rely on methods based on pattern-matching techniques which are not sustainable given the high rates at which new attack techniques are launched every day. The aim of this paper is to develop a proto-type intelligent log based intrusion detection system that can detect known and unknown intrusions automatically. Under a data mining framework, the intrusion detection system is trained with unsupervised learning algorithms specifically the k-means algorithm and the One Class SVM (Support Vector Machine) algorithm. The development of the prototype system is limited to machine generated logs due to lack of real access log files. However, the system’s development and implementation proved to be up to 85% accurate in detecting anomalous log patterns within the test logs.

Keywords


Prototype, Intrusion Detection, Log-Based, Data Mining

References