Three layer hybrid learning to improve intrusion detection system performance

Ruki Harwahyu, Fajar Henri Erasmus Ndolu, Marlinda Vasty Overbeek


In imbalanced network traffic, malicious cyberattacks can be hidden in a large amount of normal traffic, making it difficult for intrusion detection systems (IDS) to detect them. Therefore, anomaly-based IDS with machine learning is the solution. However, a single machine learning cannot accurately detect all types of attacks. Therefore, a hybrid model that combines long short-term memory (LSTM) and random forest (RF) in three layers is proposed. Building the hybrid model starts with Nearmiss-2 class balancing, which reduces normal samples without increasing minority samples. Then, feature selection is performed using chi-square and RF. Next, hyperparameter tuning is performed to obtain the optimal model. In the first and second layers, LSTM and RF are used for binary classification to detect normal data and attack data. While the third layer model uses RF for multiclass classification. The hybrid model verified using the CSE-CIC-IDS2018 dataset, showed better performance compared to the single algorithm. For multiclass classification, the hybrid model achieved 99.76% accuracy, 99.76% precision, 99.76% recall, and 99.75% F1-score.


CSE-CIC-IDS2018; Hybrid learning; Intrusion detection system; Long short-term memory; Random forest

Full Text:



Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

International Journal of Electrical and Computer Engineering (IJECE)
p-ISSN 2088-8708, e-ISSN 2722-2578

This journal is published by the Institute of Advanced Engineering and Science (IAES) in collaboration with Intelektual Pustaka Media Utama (IPMU).