Abstract
Researchers publish various studies to improve the performance of network intrusion detection systems. However, there is still a high false alarm rate and missing intrusions due to class imbalance in the multi-class dataset. This imbalanced distribution of classes results in low detection accuracy for the minority classes. This paper proposes a Synthetic Multi-minority Oversampling (SMMO) framework by integrating with a collaborative feature selection (CoFS) approach in network intrusion detection systems. Our framework aims to increase the detection accuracy of the extreme minority classes (i.e., user-to-root and remote-to-local attacks) by improving the dataset’s class distribution and selecting relevant features. In our framework, SMMO generates synthetic data and iteratively over-samples multi-minority classes. And the collaboration of correlation-based feature selection with an evolutionary algorithm selects essential features. We evaluate our framework with a random forest, J48, BayesNet, and AdaBoostM1. In a multi-class NSL-KDD dataset, the experimental results show that the proposed framework significantly improves the detection accuracy of the extreme minority classes compared with other approaches.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
The usage of computers for information sharing over network systems is tremendously increasing. Consequently, the number of security threats is also increasing with the growth of Internet applications. The computer network system suffers from security vulnerabilities since the number of network intrusions is increasing daily. Any unauthorized access to a system that violates the data’s availability, confidentiality, and integrity is an attack. As a result, different applications running on several platforms cause a serious risk to the web servers’ security [4]. For these reasons, the intrusion detection system is widely deployed to examine the events in computer and network systems. The network traffic’s automatic monitoring to efficiently identify malicious attacks from normal traffic is termed an Intrusion Detection System (IDS) [18, 43].
Network attacks’ continuous growth is becoming a severe challenge to build up adaptable cybersecurity techniques. Due to this challenge, the probability of losing data, hacking, and infringement has been grown with the Internet’s development and modernization [22]. As a result, IDSs have become an essential component for information security to protect the computer network from a particular attack. Network Intrusion Detection Systems’ (NIDS) role is to actively supervise the network’s function while detecting malicious activities [2, 4]. As network traffic gets larger, it is becoming a major problem to design and implement the NIDS to achieve the objective [59].
One of the problems of machine learning-based NIDS is missing intrusions (attacks) while learning from imbalanced data [9]. There are four main categories of attacks that include Denial of Service (DoS), probe, User-to-Root (U2R) and Remote-to-Local (R2L) [35]. DoS and probe attacks can be identified easily by careful investigation of the statistics of the connections at the vulnerable host machine. But, it is very difficult to detect the low-frequency attacks like U2R and R2L. This is because the occurrence of U2R and R2L is very rare and their number of samples are too less (i.e., extreme minority classes) when compared with the DoS and probe attacks. Therefore, the effect of data imbalance poses serious issues in intrusion detection performance [33].
Data imbalance is one of the classification problems in data mining and machine learning when certain minority classes have fewer instances than majority classes. The skewed distribution of classes could make the classifier produces adverse prediction performance for the minority classes. Various researches have suggested several solutions to solve two-class imbalanced learning [6, 7, 13, 26]. However, the two-class classification may not cover all cases in real-world applications since imbalanced multi-class data is occurring more often. The problem of multi-class data imbalance is more difficult compared with two-class cases during classification. The reason may be that multi-minority and multi-majority classes can complicate learning from imbalanced data [53].
In the domain of NIDS, studies have suggested different approaches to solve the problem of class imbalance that help in achieving better prediction performance [23, 39, 40, 43, 44, 57, 60, 61]. However, the detection accuracy of U2R and R2L attacks is still low and need to be improved.
The exact ratio of class distribution in multi-class classification is also not yet sufficiently studied in NIDS. Even though the optimal ratio of majority and minority classes is suggested to be 65% and 35% respectively for two-class classification [20], there is no clear justification for the optimal ratio of classes for multi-class classification to obtain the best prediction accuracy.
Retaining the natural distribution of classes in multi-class datasets is also an other issue to be considered while improving the general accuracy. Replication of minority class instances to balance with the majority class may not always be practical. For example, in NIDS, U2R and R2L attacks occur rarely and their number of instances is very less compared to normal, DoS and probe classes. Replicating the instances of these classes to have equal or nearly equal number of instances with the majority classes become impractical in real-world applications. Furthermore, the problem of high computational time while learning from imbalanced data in multi-class dataset needs to be solved in NIDS.
The class balancing methods like Random Over Sampling (ROS) may create over-fitting since its new instances come from the replication of original instances [53]. Synthetic Minority Oversampling Technique (SMOTE) can increase the problem of over generalization even though it decreases the risk of over-fitting. In imbalanced multi-class dataset, the generation of wrong synthetic instances can make the area of minority class to be generalized into the region of majority class. This over generalization can decrease the learnability of minority classes instances [6, 62]. In SMOTE, there is no clear agreement about the amount of percentage to be added for the instances of the minority classes during multi-class balancing. That means, by what percent do the minority classes increase in light of the size of the majority class instances to obtain the optimal class distribution?
To build robust machine learning model for NIDS, the problem of data imbalance needs to be considered. Some main issues should be examined with extensive effort both theoretically and empirically to understand in detail about the effect of imbalanced data problems for detection of attacks. The multi-class data balancing approaches have to be explored that make the machine learning algorithms to perform better when compared with learning from the skewed class distributions in NIDS. The extent of ratios for the distribution of classes while balancing from the original multi-class dataset is an other topic that needs to be studied [13].
It is known that the performance of machine learning models is largely evaluated using predictive accuracy. If the classification categories are not more or less uniformly distributed, a dataset is considered imbalanced. When the data is imbalanced, it is not easy to obtain accurate results during classification. The reason is that the classifiers favor the majority classes and the minority classes are not able to be correctly detected [16].
The motivation of our research is to propose a machine learning-based NIDS framework by integrating with the SMMO data balancing approach and collaborative feature selection for multi-class data. The framework may help balance the ratio of classes in a multi-class dataset and improve the detection accuracy of the extreme minority classes, i.e., U2R and R2L in NIDS. Furthermore, the framework integrates multi-class data balancing with the collaborative feature selection method to obtain the essential features that help improve NIDS’s general prediction performance.
This paper proposes a Synthetic Multi-minority Oversampling (SMMO) data balancing by integrating it with a collaborative feature selection (CoFS) approach for NIDS. The relevant features are selected in our work using the collaboration of the Correlation-based Feature Selection (CFS) technique and evolutionary search algorithm. This work aims to help the NIDS improve the detection accuracy of low-frequency attacks and extreme minority classes, i.e., U2R and R2L, through achieving optimal class distribution and selecting important features.
In the domain of NIDS, even though several approaches exist, to the best of our knowledge, further studies are needed to justify the optimal ratio of classes for multi-class classification. More specifically, there are no sufficient studies that show the exact ratio of classes’ distribution to obtain the best prediction accuracy for extreme minority classes in NIDS. Our work is novel in that we apply an SMMO data balancing approach to improve the classes’ degree of imbalance in NIDS with a multi-class dataset. There are no such approaches that apply the CoFS method for selecting the essential features from the balanced dataset using SMMO in NIDS. This approach helps the NIDS improve the general prediction performance and the detection accuracy of the extreme minority classes by balancing the multi-class distribution while selecting important features.
The contributions of this research to the domain of network intrusion detection systems are summarized as follows.
-
1.
We propose an SMMO to improve the degree of multi-class imbalance rather than two-class by integrating with the CoFS approach. We find the difference between majority class’s instances and other minority classes in the dataset in the proposed approach. Their difference helps decide the number of synthetic instances to be generated for each minority class during multi-class balancing.
-
2.
To select the essential features from the balanced dataset, we apply a collaborative feature selection approach that collaborates with CFS and an evolutionary search algorithm.
-
3.
We perform extensive experiments to evaluate the effectiveness of the proposed approach. The experimental results show that SMMO with
CoFS (i.e., SMMO-CoFS) produces a significant improvement of the detection accuracy for the extreme minority classes (i.e., U2R and R2L) as well as the general prediction accuracy on a multi-class intrusion dataset when compared to several other techniques.
The remaining parts of the paper are organized as follows. Section 2 summarizes related works about class balancing in the domain of network intrusion detection systems. In Sect. 3, the details of the proposed SMMO-CoFS method are presented. In Sect. 4, experiments and the results are analyzed. In Sect. 5, the discussion of the experimental results is presented including their implications. In Sect. 6, we conclude our work and outline the future works.
2 Related Works
There have been various machine learning techniques that have been published on data balancing for NIDS, such as a Synthetic Minority Oversampling Technique (SMOTE) [39, 40, 44, 57, 60], Sample Selection (SS) [5], Difficult Set Sampling Technique (DSSTE) algorithm [28] and a Data Generative Model (DGM) [9]. Deep learning techniques also have been applied to solve data imbalance in NIDs, such as the unsupervised deep learning method based on the Generative Adversarial Networks (GAN) [23], Deep Reinforcement Learning (DRL) [29], Imbalanced Generative Adversarial Network (IGAN) [14], Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) [31].
SMOTE has been applied in several types of research to solve the data imbalance issues in NIDS by integrating various machine learning techniques. Yan et al. [57] proposed a Region Adaptive (RA) SMOTE with random forest, Back Propagation Neural Network (BPNN) and Support Vector Machines (SVM) classifiers. Zhang et al. [60] applied the SMOTE combined with Edited Nearest Neighbors (SMOTE-ENN) algorithm and Convolutional Neural Network (CNN) to balance the skewed network traffic. Parsaei et al. [39] employed a hybrid approach that combines cluster center and nearest neighbor (CANN) with SMOTE for balancing the dataset. Sun et al. [46] proposed the SMOTE by integrating it with Neighborhood Cleaning Rule (NCL) for balancing the data. Liu et al. [28] propose a Difficult Set Sampling Technique(DSSTE) algorithm to tackle the class imbalance problem. In their method, they use the Edited Nearest Neighbor (ENN) algorithm to divide the imbalanced training set into the difficult set and the easy set. To compress the majority samples, they use the K-Means algorithm that help reduce the majority in the difficult set. Even though these studies improved general detection accuracy, their performance for the extreme minority classes like U2R and R2L is still very low. In addition, SMOTE can increase the problem of over-generalization though it decreases the risk of over-fitting and this over-generalization can decrease the learnability of minority classes instances [6, 62].
Despite the increasing popularity of deep learning, only some empirical works exist in the domain of NIDS to solve the problem of data imbalance. Lee et al. [23] suggested the Generative Adversarial Networks (GAN) model for solving the class imbalance in NIDS by generating new virtual data similar to the existing data. Lopez-Martin et al. [29] proposed a Radial Basis Function Neural Networks (RBFNN) for optimizing parameters within a reinforcement learning framework. The RBFNN classifier plays the role of the policy network in a Deep Reinforcement Learning (DRL) training scheme to handle classification problems with unbalanced datasets and noisy data. Huang et al. [14] proposed an Imbalanced Generative Adversarial Network (IGAN) to tackle the class imbalance problem in NIDS. They introduced an imbalanced data filter and convolutional layers to the GAN by generating new representative instances for minority classes. Meliboev et al. [31] applied the deep learning architectures of a single CNN and a combination of LSTM with SMOTE data balancing technique to improve the classification performance.
The data imbalance topic in NIDS so far focuses mainly on the extreme minority classes (i.e., U2R and R2L) during multi-classification. To increase the number of the minority classes, Phetlasy et al. [40] applied SMOTE with sequential classifier that improved sensitivity and accuracy. Abebe and Lalitha [48] synthetically oversampled the U2R and R2L attacks by 800% to improve the detection accuracy of the minority classes using random forest classifiers. However, it is not clear how they decided the amount of the oversampling percentage. Seo et al. [44] randomly generated several tuples using SMOTE to optimize the ratios of the rare classes (U2R, R2L, and probe). Each instance in the test dataset was assigned to their support vector regression model, and the best SMOTE ratios were chosen. Khor et al. [19] investigated the effectiveness of under-sampling and over-sampling methods for the rare classes problem in intrusion dataset. Their study confirmed that the two methods are less effective because of the overlapping classes in the data set. According to their study, the major learning algorithms could not generalize R2L and U2R since learning algorithms generally favor the dominant classes. Their study also showed that the overlapping classes could create difficulties in the feature selection phase. Rodda et al. [43] analyzed the class imbalance problem in intrusion datasets. In their study, they evaluated the effect of class imbalance on the benchmark NSL-KDD dataset. Their analysis was based on the four popular classification techniques (Naïve Bayes, BayesNet, J48 and random forest). Their analysis showed that none of the models achieved adequate performance for the rare class U2R.
However, still the above studies did not explain to what extent the multi-minority classes should be oversampled in relation to the majority class. The detection accuracy of the extreme minority classes is still low and need to be improved. Furthermore, they did not consider collaborative feature selection method to acquire the essential features.
3 The Proposed Framework
The purpose of our proposed SMMO-CoFS framework is to improve the class distribution of the imbalanced multi-class dataset in a network intrusion detection system. After data is balanced using SMMO, it is integrated with the CoFS approach to select important features. The integration of SMMO with CoFS is used to improve the detection accuracy of the extreme minority classes in NIDS. These extreme minority classes mainly include R2L and U2R attacks. The scheme of the proposed framework is shown in Fig. 1 and its high-level description is shown in Pseudo-code 1.
3.1 SMMO-CoFS
Even though SMOTE helps balance the binary class dataset (i.e., two categories), it has limitations to solve skewed data in multi-class datasets (i.e., multiple categories). There may be multiple minorities and multiple majority classes in some datasets. Therefore, to achieve the balanced distribution of instances in NIDS with a multi-class dataset, we propose an SMMO-CoFS. In our proposed method, SMMO calculates the difference between the number of instances of a majority class and the remaining minority classes. The calculated difference of instances between a majority class and each minority class is used to decide the number of synthetic instances to be generated. Then, each minority class is iteratively oversampled using the synthetically generated instances. After the distribution of classes in a multi-class dataset is improved, CoFS selects the important features that help enhance the general prediction performance of NIDS. As a result, SMMO-CoFS improves the detection accuracy of the extreme minority classes.
SMMO uses the following steps to obtain the balanced multi-class data set without affecting its natural class distribution [15].
-
1.
Identify the majority class and its number of class samples. NSL-KDD dataset contains five classes which are normal, Denial of Services (DoS), probe, U2R (User-to-Root) and R2L
(Remote-to-Local). Among these classes, normal class is the majority class with maximum number of class samples.
-
2.
Determine the weight of class imbalance between the majority class and the remaining minority classes.
$$\begin{aligned} w_{l}=\frac{M}{(L)\left( X_{l}\right) }, \end{aligned}$$(1)where \(w_{l}\) is the weight of class imbalance between the \(l\textrm{th}\) minority class and the majority class (w < \(\tau\), \(\tau\) is the threshold of class imbalance), M is the total number of samples in the training dataset, L is the number of unique target classes and \(X_{l}\) is the number of samples for the \(l\textrm{th}\) minority class.
-
3.
Determine the number of synthetic data instances to be created for each minority class:
$$\begin{aligned} { G } _ { l }= {J} - {X}_{l}, \end{aligned}$$(2)where \({ G } _ { l }\) is number of synthetic data instances to be created for the \(l\textrm{th}\) minority class, J is the number of majority class instances, \({X}_{l}\) is the number of the \(l\textrm{th}\) minority class instances.
-
4.
Compute the ratio of each instance \(x_{i}\) and the K nearest neighbors that we find based on the Euclidean distance:
$$\begin{aligned} r_{i}=q_{i} / K, i=1, \ldots , X, \end{aligned}$$(3)where \(q_{i}\) is the number of instances in the K nearest neighbors of \(x_{i}\) that belongs to the majority class, \(r_{i}\) \(\in\) [0, 1] .
-
5.
Normalize \(r_{i}\) according to the density distribution \(\hat{r}_{i}=r_{i} / \sum _{i=1}^{X} r_{i}\), (\(\sum _{i=1}^{X} \hat{r}_{i} = 1\)).
-
6.
Determine the number of synthetic data instances m to be created for each minority instance \(x_{i}\):
$$\begin{aligned} m_{i}=\hat{r}_{i} \times G_{l}. \end{aligned}$$(4) -
7.
Create synthetic data instances \(S_{m_{i}}\) for each minority class data instance \(x_{i}\):
$$\begin{aligned} S_{m_{i}}=x_{i}+\left( x_{\eta i}-x_{i}\right) \times \delta , \end{aligned}$$(5)where \(x_{\eta i}\) is a randomly selected instance from K-nearest neighbor, \(\delta\) is a vector in which each element is a random number, \(\delta\) \(\in\) [0,1].
-
8.
To decide the acceptance of the new synthetic samples, compare the distance of the new synthetic instances between the majority and the old minority class samples. Suppose that Q = {\({Q_{1}, Q_{2}, \dots , Q_{m}}\)} is the set of the new synthetic instances, and \(Q_{i}^{(j)}\) is the \(j\textrm{th}\) feature value of \(Q_{i}\), j \(\in\) [1, n], where n is the number of features. Let \(S_{J} = \{{S_{J1}, S_{J2}, \dots , S_{JZ}}\)} be the set of majority samples and \(S_{ml} = \{{S_{ml1}, S_{ml2}, \dots , S_{mly}}\)} be the set of samples of the \(l\textrm{th}\) minority class, then we calculate the distance between \(Q_{i}\) and \(S_{Ja}\) (i.e., \(D_\textrm{majority}(Q_{i}, S_{Ja})\)). We also calculate the distance between \(Q_{i}\) and \(S_{mlb}\) (i.e., \(D_\textrm{minority}(Q_{i}, S_{mlb})\)). For each of the newly generated synthetic instances iteratively, the distance is calculated as
$$\begin{aligned}&\begin{aligned} D_{ \textrm{majority }}\left( {Q}_{i}, S_{Ja}\right) =\\ \sum _{j=1}^{n} \sqrt{\left( {Q}_{i}^{(j)}-S_{Ja}^{(j)}\right) ^{2}}, a \in [1, Z], \end{aligned} \end{aligned}$$(6)$$\begin{aligned}&\begin{aligned} D_{ \textrm{minority }}\left( {Q}_{i}, S_{mlb}\right) =\\ \sum _{j=1}^{n} \sqrt{\left( {Q}_{i}^{(j)}-S_{mlb}^{(j)}\right) ^{2}}, b \in [1, y], \end{aligned} \end{aligned}$$(7)where \(D_{ \textrm{majority }}\left( {Q}_{i}, S_{Ja}\right)\) is the distance between the new synthetic instance \(Q_{i}\) and the majority instance \(S_{Ja}\), and \(D_{\textrm{minority }}\left( {Q}_{i}, S_{mlb}\right)\) is the distance between the new synthetic instance \(Q_{i}\) and the old minority instance \(S_{mlb}\), which is the \(l\textrm{th}\) minority class. The distance is calculated for all newly created synthetic data instances with each minority class. Based on the distance, if the minimum value of each \(D_\textrm{minority}\) is less than the minimum value of \(D_\textrm{majority}\), then the new synthetic instance is accepted otherwise, it is rejected.
-
9.
Determine the percentage of the synthetic samples to be added for each minority class. We use ceiling function to find the integral multiple of 100 and defined as
$$\begin{aligned} {A}_{l} = \left\lceil \frac{S_{m_{i}}}{100} \right\rceil , \end{aligned}$$(8)where \({A}_{l}\) is the number of class samples to be added for the \(l\textrm{th}\) minority class.
-
10.
Iteratively oversample each minority class using the generated synthetic instances
$$\begin{aligned} S_{m_{l}}=X_{l}+A_{l} \times \delta , \end{aligned}$$(9)where \(S_{m_{l}}\) is the \(l\textrm{th}\) minority class, which is synthetically oversampled, and \(X_{l}\) is the old number of instances for the \(l\textrm{th}\) minority class.
3.2 Collaborative Feature Selection (CoFS)
In the proposed framework, we use the collaboration of the Evolutionary Algorithm (EA) and Correlation Feature Selection (CFS) technique to select the important features from the improved dataset in its class distribution (i.e., balanced dataset). Using EA, we find the best solution from the population of attributes. Then CFS evaluates the obtained subset of the solution to select the most important features subset for classification purposes.
The advantage of EA over others is it allows the best solution to emerge from the best of prior solutions. It improves the selection of features over time. EA creates new and more fitted individuals by combining the generation of the different solutions and extracts the best genes (features) from each one [12, 34]. Additionally, the worth of a subset of attributes is evaluated using the CFS technique. CFS considers the individual predictive ability of each feature along with the degree of redundancy between them.
Collaborative feature selection uses the following steps to select useful features from the balanced dataset.
-
1.
Encode features (solutions to the problem) using EA with a constant length vector of bits where \(x(i)=1\) denotes that EA selects the \(i\textrm{th}\) feature and \(x(i) =0\) means the opposite.
-
2.
Make a random initialization in EA to populate the initial population with completely random features.
-
3.
Determine the fitness function (evaluation function) to check how fit a solution is. It evaluates how close a given solution subset is to the optimum number of features for NIDS. To assess the fitness of each feature from the solution subsets, we use the CFS subset evaluation method. The feature that is highly correlated with the class is the fittest. It is described as follows [55].
$$\begin{aligned} f_{t}=\frac{n r_{c f}}{\sqrt{n+n(n-1) r_{f f}}}, \end{aligned}$$(10)where \(f_{t}\) is the fitness score of the feature (gene) in the population, f is the feature in one of the solution subsets (chromosome), n is the number of features in each solution subsets, \(r_{cf}\) is the mean of class-feature correlations and \(r_{ff}\) is the mean of feature–feature correlations in the obtained solution subsets from the population.
-
4.
Determine the selection operation for EA. In this research, we use binary tournament selection. Its efficiency for both non-parallel and parallel architecture makes this selection technique preferable to random and rank selection techniques. It does not require sorting the whole population, and its selection pressure can be adjusted easily. In T-way tournament selection, it selects T features from the population at random and chooses the best out of them to become a parent. The same processes are repeated for selecting the next parent feature [1, 41].
-
5.
Determine the crossover operation for EA. We use single-point crossover since it is probably one of the simplest crossover operators. A random crossover point is selected in this operator, and the tails of its two parents are swapped to get new off-spring features. The probability of crossover can be described as [50, 56]:
$$\begin{aligned} P_{c}=\left\{ \begin{array}{c} \frac{P_{c}\left( f_{t_{\max }}-F\right) }{f_{t_{\max }}-f_{t_{\min }}}, \quad F \ge f_{t_{\text{ avg } }} \\ P_{c 1}, \quad F<f_{\text{ tavg } }, \end{array}\right. \end{aligned}$$(11)where \(P_{c}\) is the probability of crossover, F is the value of the more adaptable parent
employed for crossover, \(F_{t_\textrm{min}}\) is the smallest value of fitness obtained from the population, \(f_{t_\textrm{max}}\) is the greatest value of fitness in the population, \(P_{c1}\) is the numbers whose values are within the range of (0,1) and \(f_{t_\textrm{avg}}\) is the value of average fitness obtained from the population.
-
6.
Determine the mutation operation for EA. We use a bit-flip mutation operator to optimize functions over binary strings. The bit-flip mutation acts independently on each bit in a solution. This operator changes the value of the bit (i.e., from 0 to 1 and vice versa) with the probability of mutation \(P_{m}\) and it is described as [45]:
$$\begin{aligned} P_{m}=\left\{ \begin{array}{c} \frac{P_{m 1}\left( f_{t \max }-F\right) }{f_{t_{\max }}-f_{t_{\min }}}, \quad F \ge f_{t_{{\text {avg}}}} \\ P_{m 1}, \quad F<f_{t_{\text{ avg } }}, \end{array}\right. \end{aligned}$$(12)Where F is the value of the more adaptable parent employed for mutation, \(f_{t_\textrm{min}}\) is the smallest value of fitness obtained from the population, \(f_{t_\textrm{max}}\) is the greatest value of fitness in the population, \(P_{m1}\) is the numbers whose values are within the range of (0, 1) and \(f_{t_\textrm{avg}}\) is the value of average fitness obtained from the population.
-
7.
Determine the survivor selection technique to identify the best subset of solutions (features). We use elitism-based survivor selection (generational replacement) since it is more efficient to improve the performance of EA. Elitism-based survivor selection determines which features are to be discarded and which are to be kept in the next generation. In this selection technique the best subset of solutions (features) are always kept [58].
3.3 Deployment of SMMO-CoFS-Based NIDS
Even though Machine Learning (ML)-based network intrusion detection systems can take the network traffic (input) to predict a network behavior, these systems still cannot be directly implemented in real-time network environments. The ML-based models do not have packet sniffers that capture the network traffic in real-time. A network sniffer (packet sniffer) is a traffic monitoring and analysis tool that can identify (sniff out) the data flowing over the network in real-time. For achieving real-time detection, the developed ML-based network intrusion detection systems have to work with packet sniffers, such as Spark, Bro, and Snort [25].
When our proposed SMMO-CoFS-based NIDS is deployed, the packet sniffer captures the network traffic and will be pre-processed. SMMO-CoFS is used to pre-process the data. The processed data will be used as input for the prediction decision of the classifier. Finally, the classifier predicts one of the five target classes, including the normal traffic data and one of the four attacks (i.e., DoS, probe, R2L and U2R). The network administrators will decide the required network intrusion prevention mechanisms based on the report. The deployment of SMMO-CoFS-based NIDS is as shown in Fig. 2.
4 Experiments and Result Analysis
4.1 Benchmark Dataset
We conduct our experiment using Network Security Laboratory-Knowledge Discovery and Data Mining(NSL-KDD) dataset to evaluate our proposed method. NSL-KDD dataset is an effective benchmark data set to help researchers compare different intrusion detection methods despite a lack of public data sets for network-based IDSs. The data set, NSL-KDD, comprises 41 features with five classes labeled as attacks type (intrusions) and normal. The intrusions are categorized into four classes of attacks, which are: Denial of Services (DoS), Probe, User-to-Root (U2R), and Remote-to-Local (R2L) [30, 37, 47].
4.2 Experimental Framework
In our experiment, to test our proposed method’s prediction performance, we make the partition of data by applying 10-fold cross-validation. Then we build models on four learners viz. random forest, J48, BayesNet, and AdaBoostM1. We also carry out the statistical significance test to evaluate the significance of the SMMO-CoFS method’s improved performance. This significance test is evaluated against other methods using the Wilcoxon statistical significance test. After conducting an extensive experiment, we also compare our proposed method’s performance with other methods previously proposed by other researchers.
RapidMiner Studio Professional 9.6 [32], WEKA version 3.8 [52], KEEL (Knowledge Extraction based on Evolutionary Learning) software tool [49], OriginPro 2018 [38], and IBM SPSS Statistic 23 [36] are adopted to create and examine the empirical results.
4.3 Evaluation Metrics
4.3.1 Prediction Performance Test
The evaluation of performance in our experiment is conducted by using the evaluation metrics: True Positive Rate (TPR), Precision (Pr), Recall (Rc), F Measure (F1), and Receiver Operating Characteristic (ROC) curve and time (s). The degree of imbalance (mild, moderate and extreme), and the proportion of minority classes is also considered. Since the classification is multi-class, for each classes l, the number of classes L and the micro average \(\mu\), the formulas to calculate the evaluation metrics are given as follows [17, 18]. The description of the notations which are used in the formulas is also shown in Table 1.
4.3.2 Statistical Significance Test
To prove that the performance improvement of the SMMO-CoFS method is statistically significant, we use the mostly used Wilcoxon statistical significance test [8, 24]. The test is evaluated against the original data, the CoFS (no data balancing) and the SMOTE method. We also make a statistical test to prove how our proposed method significantly improves the results of the existing related research works.
The Wilcoxon signed-ranks test is a nonparametric statistical test that ranks the differences in performances of two feature selection methods using the dataset. It ignores the signs and compares the ranks for the positive and the negative differences. Let \(d_{i}\) be the difference between the performance scores of the feature selection methods on the \(i\textrm{th}\) classification model; then the differences are ranked according to their absolute values. The average ranks are assigned in case of ties. Let \(R^{+}\) be the sum of ranks on which the second algorithm outperformed the first, and \(R^{-}\) the sum of ranks for the opposite, then
and
Ranks of \(d_{i} = 0\) are split evenly among the sums. If there is an odd number of them, one is ignored. Let the Wilcoxon test \(\tau\) be the smaller of the sums, then \(\tau\) = \(\textrm{min}(R^{+},R^{-})\) for confidence level of \(\alpha = 0.05\).
4.4 Result Analysis
4.4.1 Analysis of Degree of Imbalance for Multi-minority Classes
Table 2 and Fig. 3 show the distribution of instances for the five classes (i.e., normal, DoS, probe, U2R and R2L) for the original training dataset and after applying the proposed SMMO-CoFS method. As shown in Table 2, the minority class’s degree of imbalance, in the original training dataset, is mild for DoS, moderate for probe, and extreme for U2R and R2L [21] [42]. After the SMMO-CoFS method is applied, the distribution of instances for each class is improved. As shown in Table 2 and Fig. 3, the proportion of DoS class is slightly decreased from 36.45 to 36% while the proportion of probe, U2R and R2L is increased. The proportion of probe, U2R and R2L is increased from 9.25 to 9.6%, 0.04 to 0.6%, and 0.78 to 1.3%, respectively. As a result, the degree of imbalance for the R2L class is improved from extreme to moderate. The proportion of the majority class (i.e., normal class) is decreased from 53.45 to 52.5%, which improves the minority class’s degree of imbalance in the dataset by keeping the natural distribution of attacks.
4.4.2 Analysis of Recall and F measure
In Tables 3 and 4, we report the detailed performance results of the proposed SMMO-CoFS compared to the original dataset and the SMOTE method in terms of True Positive Rate (TPR), Precision (Pr), Recall (Rc), F measure (F1), and ROC curve. These tables describe the SMMO-CoFS’s evaluation results for predicting the two extreme minority classes: U2R and R2L. Random forest, J48, BayesNet, and AdaBoostM1 classifiers are used for evaluating the performances. As shown in Tables 3 and 4, the performance of the SMMO-CoFS method outperforms the original data set and SMOTE in terms of all evaluation metrics (i.e., TPR, Pr, Rc, F-measure, and ROC curve) for the U2R class. SMMO-CoFS also achieves the improved performance for R2L in terms of ROC with J48 and Pr, F1 and ROC with BayesNet. The results in these tables also show that the proposed SMMO-CoFS method achieves the best performance with a random forest classifier in terms of most metrics. Furthermore, the results show a significant performance improvement for the extreme minority classes, especially for U2R, in terms of the indicated evaluation metrics. In Tables 3 and 4, the performance of SMMO-CoFS outperforms the original data set and SMOTE in terms of Rc and F1 for U2R class with all classifiers. It also achieves the improved performance in terms of F1 for R2L class with the BayesNet classifier.
In Tables 5 and 6, we report the detailed performance results of the proposed SMMO-CoFS method compared to CoFS (i.e., without data balancing) in terms of TPR, Pr, Rc, F1, and ROC curve for predicting U2R and R2L. We use the random forest, J48, BayesNet, and AdaBoostM1 classifiers for evaluating the performances. As shown in Tables 5 and 6, the SMMO-CoFS method achieves the best performance compared with the CoFS in terms of most evaluation metrics (i.e., TPR, Pr, Rc, F1, and ROC curve). The results in these tables also show that the proposed SMMO-CoFS method achieves the best performance with a random forest classifier in terms of most metrics. Furthermore, the results show a performance improvement for both extreme minority classes U2R and R2L in terms of most evaluation metrics.
4.4.3 Analysis of True-Positive Rate (TPR)
Figure 4 shows the performance of the proposed SMMO-CoFS method in terms of TPR with the four classifiers (i.e., random forest, J48, BayesNet and AdaBoostM1) for predicting U2R and R2L classes. The experimental results are taken from Tables 3, 4, 5 and 6. The results in this figure also show the performance of the four classifiers with the original dataset, CoFS and SMOTE. As shown in Fig. 4, the performance of the proposed SMMO-CoFS method outperforms the other methods using the four classifiers in terms of TPR. The proposed method also significantly improves the prediction performance of the indicated classifiers for U2R in terms of TPR, which is the extreme minority class.
4.4.4 Analysis of Precision
Figure 5 shows the performance of the proposed SMMO-CoFS method in terms of precision with the four classifiers (i.e., random forest, J48, BayesNet and AdaBoostM1). The experimental results are taken from Tables 3, 4, 5 and 6. The results in this figure also show the performance of the four classifiers with the original dataset, CoFS and SMOTE. As shown in Fig. 5, the proposed SMMO-CoFS method outperforms the other methods with the four classifiers for U2R and R2L classes in terms of Pr. The proposed method also significantly improves the prediction performance of the indicated classifiers for U2R in terms of Pr, which is the extreme minority class.
4.4.5 Analysis of ROC Curve
Figure 6 and 7 show the performance of the SMMO-CoFS method and the original dataset in terms of ROC curves. The ROC curves are constructed by plotting the True Positive Rate (TPR) against the False Positive Rate (FPR) of random forest classifier, which achieves the best performance as shown in Tables 3 and 4. Figure 6 shows the ROC curves of the two extreme minority classes (i.e., U2R and R2L) using the SMMO-CoFS method. As shown in this table, the classes’ ROC curves are aligned or very close to the upper-left corner of the graph, which shows that the results are very close to perfect classification. As shown in Fig. 7, the original dataset ROC curves are relatively far from the graph’s upper-left corner, which shows that the random forest classifier with the original dataset achieves less classification accuracy. Therefore, the proposed SMMO-CoFS method helps the classifier achieve efficient and accurate prediction for the extreme minority classes (i.e., U2R and R2L) in terms of ROC curve.
4.4.6 Analysis of Time Complexity
Table 7 shows the performance evaluation of SMMO-CoFS method in terms of time with the four classifiers (i.e., random forest, J48, BayeNet and AdaBoostM1). The table also includes the the performance of classifiers with the original dataset, CoFS and SMOTE. As shown in this table, the proposed SMMO-CoFS method achieves faster processing speed with the four classifiers. The experimental results in Table 7 also show that the processing speed of all classifiers is higher (i.e., it takes slow processing time) with the SMOTE. Furthermore, the four classifiers with CoFS without data balancing achieves medium speed (time) (i.e., it is faster than the original dataset and SMOTE, but slower than the SMMO-CoFS).
Figure 8 shows the performance of the proposed SMMO-CoFS, original dataset, CoFS and SMOTE using the four classifiers (random forest, J48, BayesNet and AdaBoostM1) in terms of processing time. The experimental results are taken from Table 7. As shown in this figure, the processing time of the proposed SMMO-CoFS method is less (faster) than CoFS, SMOTE and the original dataset. Furthermore, Fig. 8 shows that the processing time of SMMO-CoFS method is faster with BayesNet classifier. On the other hand, the processing time of SMMO-CoFS method is slower with random forest classifier though better than the other methods.
4.4.7 Analysis of Statistical Significance Test
The results of Wilcoxon signed statistical significance test is shown in Table 8 that compares the performance of the proposed SMMO-CoFS method using the four classifiers with the original data, CoFS and SMOTE methods. The statistical significance test is conducted to confirm that our proposed method’s performance difference is statistically significant in terms of TPR for the extreme minority class U2R. This test includes random forest, J48, BayesNet, and AdaBoostM1 classifiers with the corresponding methods. Table 9 shows the results of the Wilcoxon signed statistical significance test to compare the proposed SMMO-CoFS method with the current research results in terms of accuracy and F measure.
In Tables 8 and 9, the P-values of all comparing methods are significant, i.e., P < 0.05 with confidence interval level of \(\alpha\) = 0.05. The test statistic \(\textrm{min}\)(\(R^{+},R^{-}\)) is smaller and their P-values are less than the threshold \(\alpha\) = 0.05. This statistical significance test shows that our proposed SMMO-CoFS method outperforms the other techniques and the original data’s performance. Therefore, the performance improvement that is achieved by our proposed method is statistically significant.
Table 10 shows the result of a statistical t-Test using SAS (Statistical Analysis Software) to prove that there is a significant difference between the means of the processing time for the proposed SMMO-CoFS method and the original dataset, CoFS and SMOTE methods. The test controls the Type I comparison-wise error rate with (\(\alpha\) = 0.05, Least Significant Difference (LSD) = 14.072, Error Degrees of Freedom (EDF) = 9, Error Mean Square (EMS) = 77.394, Critical Value of t (CV-t) = 2.262). As shown in Table 10, the t-grouping letters (i.e., A, B and C) indicate the difference and similarity of the mean values of the comparing methods. The mean values with the same letter are not significantly different. Table 10 shows that the proposed SMMO-CoFS method has different t-grouping compared to the original data and SMOTE method. Therefore, even though the processing time of the proposed method is not significantly different from CoFS technique, it achieves a statistically significant difference (improvement) when compared with the original data and SMOTE method.
4.4.8 Performance Comparison
The performance comparisons of our proposed method’s result with the existing current research works are presented in Tables 11 and 12. Table 11 exhibits some of the lists of NIDS frameworks suggested by different authors in previous researches. This table also demonstrates the comparison of the proposed SMMO-CoFS-based NIDS framework with other frameworks in terms of detection rate for the extreme minority classes (U2R and R2L). The frameworks are chosen since their problem domain is NIDS with machine learning topics. Furthermore, all the compared frameworks have been evaluated using the NSL-KDD dataset similar to our dataset.
As shown in Table 11, our proposed framework outperforms all other frameworks with the indicated performance evaluation metric. The evaluation metric detection rate, is chosen since it is used by most of the researchers. Therefore, the comparison results show that the proposed SMMO-CoFS method helps the NIDS achieve improved classification performance for the extreme minority classes compared with the other methods.
Table 12 shows the performance comparison of the proposed SMMO-CoFS method with the results of current existing related research works in terms of general accuracy and F-measure. The results in Table 12 proves that our proposed SMMO-CoFS method outperforms the other methods in terms of accuracy and F-measure.
5 Discussion and Implications
5.1 Discussion
5.1.1 SMMO-CoFS Method in Improving Classes’ Degree of Imbalance
The experiment results in Table 2 and Fig. 3 show that SMMO-CoFS helps in improving the proportion of multi-class NIDS dataset and the minority classes’ degree of imbalance. Consequently, these results confirm that SMMO-CoFS helps in retaining the NIDS’s natural class distribution while improving sclasses. In addition, Figs. 6 and 7 show that data balancing helps in achieving best prediction performance when the measurement is in terms of ROC curves as explained in [6, 54].
5.1.2 SMMO-CoFS Method in Improving Detection Accuracy of Multi-minority Classes
The experimental results shown in Tables 3, 4, 5, 6, 8, 11, 9 and 12 as well as in Figs. 4, 5, 6 and 7 confirm that integrating multi-class data balancing with feature selection method helps the NIDS in improving the detection performance of the minority classes. Furthermore, the results assert that the integration of synthetic multi-minority oversampling and collaborative feature selection method improves the NIDS’s performance in detecting the extreme minority classes (i.e., U2R and R2L attacks). As a result, the proposed SMMO-CoFS method significantly improves the detection accuracy of these extreme minority classes as well as the general prediction performance of NIDS in multi-class dataset.
5.1.3 SMMO-CoFS Method in Selecting Useful Features
As shown in Table 7, the proposed SMMO-CoFS method helps the NIDS in selecting the essential features which contribute for improving the prediction performance. The experimental results shown in Tables 3, 4, 5 and 6 confirm that SMMO-CoFS outperforms CoFS in terms of most of the evaluation metrics. These results ascertain that SMMO-CoFS helps in selecting relevant and non-redundant features instead of using the dataset with skewed distribution of classes. As a result, the selected features further improve the prediction performance of classifiers to detect the extreme minority classes in NIDS.
5.1.4 SMMO-CoFS Method in Improving Processing Time
The experimental results presented in Tables 7 and 10, and Fig. 8 affirm that the proposed SMMO-CoFS achieves faster processing time. The experimental results assures that multi-class data balancing significantly helps the NIDS in achieving faster processing time while improving prediction performance. Furthermore, the results confirm that achieving both faster processing time and improved prediction performance is difficult. As shown in Table 7 and Fig. 8, random forest classifier achieves slower processing time when compared with other classifiers with SMMO-CoFS method. But random forest achieves the best prediction performance as shown in Tables 3, 4, 5 and 6 using the proposed SMMO-CoFS method. On the other hand, ByesNet classifier achieves faster processing time but its prediction performance is less than random forest’s performance with SMMO-CoFS method.
5.2 Implications
The integration of multi-class data balancing and collaborative feature selection methods help data scientists during data cleaning and pre-processing in NIDS. Therefore, the proposed SMMO-CoFS method certainly helps experts select the most appropriate and optimum number of features. It contributes to improving the prediction performance of machine learning models during data analysis tasks.
SMMO-CoFS method likely helps machine learning engineers develop and implement robust NIDS to detect the extreme minority classes (i.e., low-frequency attacks). For machine learning-based NIDS model developers, applying the SMMO-CoFS method certainly helps in building efficient NIDS. This NIDS contributes to network administrators effectively identifying, monitoring, and analyzing network traffic to protect a system from possible attacks.
In general, NIDS based on the SMMO-CoFS method certainly supports organizations in building high performance, secured, reliable and scalable network. It also helps companies in identifying any unwarranted networking threats before the intrusion penetrates into a full-force security attack, data leak, and service outages. Furthermore, the SMMO-CoFS method likely improves organizations’ business models and services by transforming large data sets into knowledge and actionable intelligence.
5.3 Limitations of the Study
Even though evolutionary algorithms have been demonstrated to be an effective approach for solving real-world optimization problems, they can not guarantee the optimal result in the presence of huge solution space [10]. In addition, the limitation of CFS is its failure to select features that have locally predictive values when overshadowed by strong, globally predictive features [11].
It would be interesting to apply a feature selection and instance selection concurrently with the combination of Reinforcement Learning (RL) and Deep Learning (DL) algorithms. The agent in RL may find an optimal classification policy in imbalanced data under the guidance of a specific reward function and beneficial learning environment. The approach may help select important features while acquiring dimensionality reduction and improving efficiency for NIDS [3, 51].
6 Conclusion
This paper has demonstrated the positive impact of integrating Synthetic Multi-minority Oversampling (SMMO) with collaborative feature selection (CoFS) to increase the detection accuracy of the extreme minority classes in network intrusion detection systems. SMMO-CoFS improves the distribution of classes in a multi-class dataset by minimizing the degree of imbalance for the multi-minority classes. As a result, the proportion of the extreme minority classes is improved. The ratio of instances for R2L is improved to moderate, and the ratio of U2R is increased in the multi-class NSL-KDD dataset. The study also indicates that applying a collaborative feature selection approach helps select relevant features while reducing dimensionality. Consequently, the proposed SMMO-CoFS method significantly improves the detection accuracy of these extreme minority classes in multi-class NIDS dataset.
In this work, we have compared our framework’s performance with the original dataset, collaborative feature selection method (i.e., without applying data balancing) and SMOTE. The proposed framework achieves the best classification performance using random forest, J48, BayesNet, and AdaBoostM1. The statistical significance of the detection accuracy improvement achieved by the proposed framework is tested. It is proved that the difference of improvement in terms of detection rate is statistically significant. Our proposed NIDS framework is also compared with other researchers proposed approaches, and it achieves the best performance in detecting the extreme minority classes (i.e., U2R and R2L attacks) as well as the general prediction performance of the NIDS.
The experiment results provide compelling evidence and suggest that the integration of SMMO with a collaborative feature selection helps achieve improved class distribution while selecting essential features. As a result, the framework contributes to improving the detection accuracy of the extreme minority classes in network intrusion detection systems. Therefore, the proposed framework is reliable for NIDSs, and it is a valuable method in achieving the improved detection performance of multi-minority classes.
In the future, deep learning approaches will be explored to build more effective and robust IDSs in computer network systems. Furthermore, we will explore deep learning to build stronger NIDS in Internet of Things (IoT) systems. It may help detect Distributed Denial of Service (DDoS) attack across the IoT systems.
Availability of data and material
The data that support the findings of this study are openly available in Canadian Institute for Cybersecurity datasets at https://www.unb.ca/cic/datasets/nsl.html.
Abbreviations
- BPNN:
-
Back propagation neural network
- CANN:
-
Cluster center and nearest neighbor
- CFS:
-
Correlation-based feature selection
- CNN:
-
Convolutional neural network
- CoFS:
-
Collaborative feature selection
- CV-t :
-
Critical value of t
- DGM:
-
Data generative model
- DoS:
-
Denial of service
- DRL:
-
Deep reinforcement learning
- DSSTE:
-
Difficult set sampling technique
- EA:
-
Evolutionary algorithm
- ENN:
-
Edited nearest neighbors
- EDF:
-
Error degrees of freedom
- EMS:
-
Error mean square
- FN:
-
False negative
- FP:
-
False positive
- FPR:
-
False-positive rate
- GAN:
-
Generative adversarial networks
- IDS:
-
Intrusion detection system
- IG:
-
Information gain
- IGAN:
-
Imbalanced generative adversarial networks
- IoT:
-
Internet of things
- KEEL:
-
Knowledge extraction based on evolutionary learning
- KL:
-
Kullback–Leibler
- LSD:
-
Least significant difference
- LOO:
-
Leave one out
- LSTM:
-
Long-short term memory
- ML:
-
Machine learning
- NCL:
-
Neighborhood cleaning rule
- NIDS:
-
Network intrusion detection system
- NSL-KDD:
-
Network security laboratory-knowledge discovery and data mining
- R2L:
-
Remote-to-local
- RA-SMOTE:
-
Region adaptive synthetic minority oversampling technique
- RBFNN:
-
Radial basis function neural network
- ROC:
-
Receiver operating characteristic
- SAS:
-
Statistical analysis software
- SMMO:
-
Synthetic multi-minority oversampling
- SMOTE:
-
Synthetic minority oversampling technique
- SS:
-
Sample selection
- SVM:
-
Support vector machines
- TN:
-
True negative
- TP:
-
True positive
- TPR:
-
True-positive rate
- U2R:
-
User-to-root
References
Abd Rahman, R., Ramli, R., Jamari, Z., et al.: Evolutionary algorithm with roulette-tournament selection for solving aquaculture diet formulation. Math. Probl. Eng. Math. Probl. Eng. 2016, 1–10 (2016)
Amrita, K.K.R.: A hybrid intrusion detection system: Integrating hybrid feature selection approach with heterogeneous ensemble of intelligent classifiers. Int. J. Netw. Secur. 20, 41–55 (2018)
Asharf, J., Moustafa, N., Khurshid, H., et al.: A review of intrusion detection systems using machine and deep learning in internet of things: Challenges, solutions and future directions. Electron 9, 1177 (2020)
Buczak, A.L., Guven, E.: A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun. Surv. Tuts 18, 1153–1176 (2016)
Chaïri, I., Alaoui, S., Lyhyaoui, A.: Balancing distribution of intrusion detection data using sample selection. J. Inf. Secur. Res. 3, 153–163 (2012)
Chawla, N.V., Bowyer, K.W., Hall, L.O., et al.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)
Chen, S., He, H., Garcia, E.A.: Ramoboost: ranked minority oversampling in boosting. IEEE Trans. Neural Netw. 21, 1624–1642 (2010)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. ML Res. 7, 1–30 (2006)
Dlamini, G., Fahim, M.: Dgm: a data generative model to improve minority class presence in anomaly detection domain. Neural Comput. Appl. 2021, 1–12 (2021)
D’Angelo, G., Palmieri, F.: Gga: a modified genetic algorithm with gradient-based local search for solving constrained optimization problems. Inf. Sci. 547, 136–162 (2021)
Hall, MA.: Correlation-based feature selection of discrete and numeric class machine learning. Working Paper Series 2000:1–9 (2000)
Hassanat, A., Almohammadi, K., Alkafaween, E., et al.: Choosing mutation and crossover ratios for genetic algorithms-a review with a new dynamic approach. Information 10, 1–36 (2019)
He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21, 1263–1284 (2008)
Huang, S., Lei, K.: Igan-ids: an imbalanced generative adversarial network towards intrusion detection system in ad-hoc networks. Ad Hoc Netw. 105(102), 177 (2020)
Hussein, A.S., Li, T., Yohannese, C.W., et al.: A-smote: A new preprocessing approach for highly imbalanced datasets by improving smote. Int. J. Comput. Intell. Syst 12, 1412–1422 (2019)
Ibrahim, M.H.: Odbot: outlier detection-based oversampling technique for imbalanced datasets learning. Neural Comput. Appl. 33, 1–26 (2021)
Ji, H., Wang, Y., Qin, H., et al.: Comparative performance evaluation of intrusion detection methods for in-vehicle networks. IEEE Access 6, 37523–37532 (2018)
Keserwani, P.K., Govil, M.C., Pilli, E.S.: An effective nids framework based on a comprehensive survey of feature optimization and classification techniques. Neural Comput. Appl. 2021, 1–21 (2021)
Khor, K.C., Ting, C.Y., Phon-Amnuaisuk, S.: The effectiveness of sampling methods for the imbalanced network intrusion detection data set. In: Recent Advances on Soft Computing and Data Mining: Proceedings of the First International Conference on Soft Computing and Data Mining, pp. 613–622 (2014)
Khoshgoftaar, T.M., Seiffert, C., Van Hulse, J., et al.: Learning with limited minority class data. In: Proc. 6th Int. Conf. Machine Learning Applications (ICMLA), pp. 348–353 (2007)
Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: Comparing boosting and bagging techniques with noisy and imbalanced data. IEEE Trans. Syst. Man Cybern. Syst. 41, 552–568 (2010)
Kumar, K., Batth, J.S.: Network intrusion detection with feature selection techniques using machine-learning algorithms. Int. J. Comput. Appl. 150, 0975–8887 (2016)
Lee, J., Park, K.: Gan-based imbalanced data intrusion detection system. Pers. Ubiquitous Comput. 2019, 1–8 (2019)
Lee, J., Seo, W., Kim, D.W.: Effective evolutionary multilabel feature selection under a budget constraint. Complexity 2018, 1–14 (2018)
Li, J., Qu, Y., Chao, F., et al.: (2019) Machine learning algorithms for network intrusion detection. AI Cyb. Sec. 1, 151–179 (2019)
Lim, P., Goh, C.K., Tan, K.C.: Evolutionary cluster-based synthetic oversampling ensemble (eco-ensemble) for imbalance learning. IEEE Trans. Cybern. 47, 2850–2861 (2016)
Lin, W.C., Ke, S.W., Tsai, C.F.: Cann: an intrusion detection system based on combining cluster centers and nearest neighbors. Knowl-Based Syst. 78, 13–21 (2015)
Liu, L., Wang, P., Lin, J., et al.: Intrusion detection of imbalanced network traffic based on machine learning and deep learning. Ieee Access 9, 7550–7563 (2020)
Lopez-Martin, M., Sanchez-Esguevillas, A., Arribas, J.I., et al.: Network intrusion detection based on extended rbf neural network with offline reinforcement learning. IEEE Access 9, 153153–153170 (2021)
Tavallaee, M., Bagheri, WLE., Ghorbani, A.: NSL-KDD dataset. https://www.unb.ca/research/iscx/dataset/ iscx-NSL-KDD-dataset.html. Accessed 2 June 2020 (2020)
Meliboev, A., Alikhanov, J., Kim, W.: Performance evaluation of deep learning based network intrusion detection system across multiple balanced and imbalanced datasets. Electronics 11(4), 515 (2022)
Mierswa, I.: RapidMiner Studio. https://rapidminer.com/products/studio/ Accessed 2 June 2020 (2020)
Mishra, P., Varadharajan, V., Tupakula, U., et al.: A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Commun. Surv. Tuts 21, 686–728 (2018)
Moslehi, F., Haeri, A.: An evolutionary computation-based approach for feature selection. J. Ambient Intell. Humaniz. Comput. 11, 1–13 (2019)
Nisioti, A., Mylonas, A., Yoo, P.D., et al.: From intrusion detection to attacker attribution: a comprehensive survey of unsupervised methods. IEEE Commun. Surv. Tuts 20, 3369–3388 (2018)
Norman H. Nie, D.H.B., Hull, C.H.: IBM SPSS Statistics-23. https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-23. Accessed 2 June 2020 (2020)
Olusola, A.A., Oladele, A.S., Abosede, D.O.: Analysis of kdd99 intrusion detection dataset for selection of relevance features. In: Proc. World Congress Engineering Computer Science (WCECS), pp 20–22 (2010)
OriginLab, C.: Origin2018Pro. https://www.originlab.com/index.aspx?go=Support &pid=3301 Accessed 2 June 2020 (2020)
Parsaei, M.R., Rostami, S.M., Javidan, R.: A hybrid data mining approach for intrusion detection on imbalanced nsl-kdd dataset. Int. J. Adv. Comput. Sci. Appl. 7, 20–25 (2016)
Phetlasy, S., Ohzahata, S., Wu, C.: et al Applying smote for a sequential classifiers combination method to improve the performance of intrusion detection system. In: IEEE Int. Conf. on Dependable, Autonomic and Secure Computing, Int. Conf on Pervasive Intelligence and Computing, Int. Conf on Cloud and Big Data Computing, Int. Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), pp 255–258 (2019)
Prayudani, S., Hizriadi, A., Nababan, E., et al Analysis effect of tournament selection on genetic algorithm performance in traveling salesman problem (tsp). J. Phys. Conf. Ser., pp 1–7 (2020)
Procopio, M.J., Mulligan, J., Grudic, G.: Coping with imbalanced training data for improved terrain prediction in autonomous outdoor robot navigation. In: IEEE Int. Conf. Robotics and Automation, pp. 518–525 (2010)
Rodda, S., Erothi, USR.: Class imbalance problem in the network intrusion detection systems. In: Int. Conf. on Electrical, Electronics, and Optimization Techniques (ICEEOT), pp. 2685–2688 (2016)
Seo, J.H., Kim, Y.H.: Machine-learning approach to optimize smote ratio in class imbalance dataset for intrusion detection. Comput. Intell. Neurosci. 2018, 1–11 (2018)
Shukla, A.K.: Detection of anomaly intrusion utilizing self-adaptive grasshopper optimization algorithm. Neural Comput. Appl. 2020, 1–21 (2020)
Sun, Y., Liu, F.: Smote-ncl: a re-sampling method with filter for network intrusion detection. In: 2nd IEEE Int. Conf. Computer Communications (ICCC), pp. 1157–1161 (2016)
Tavallaee, M., Bagheri, E., Lu, W., et al.: A detailed analysis of the kdd cup 99 data set. In: IEEE Symp. Computational Intelligence for Security and Defense Applications, pp. 1–6 (2009)
Tesfahun, A., Bhaskari, D.L.: Intrusion detection using random forests classifier with smote and feature reduction. In: Proc. Int. Conf. Cloud Ubiquitous Computing Emerging Technologies, pp. 127–132 (2013)
Triguero, SG., J.M. Moyano, S.G.: KEEL software tool. http://www.keel.es/. Accessed 2 June 2020 (2020)
Varun Kumar, S., Panneerselvam, R.: A study of crossover operators for genetic algorithms to solve vrp and its variants and new sinusoidal motion crossover operator. nt J Comput Intell Res 13, 1717–1733 (2017)
Vinayakumar, R., Alazab, M., Soman, K., et al.: Deep learning approach for intelligent intrusion detection system. Ieee Access 7, 41525–41550 (2019)
Waikato, U.: WEKA: the workbench for machine learning. http://www.cs.waikato.ac.nz/ml/index. html. Accessed 2 June 2020 (2020)
Wang, S., Yao, X.: Multiclass imbalance problems: analysis and potential solutions. IEEE Trans. Syst. Man Cybern. Syst. 42, 1119–1130 (2012)
Weiss, G.M., Provost, F.: Learning when training data are costly: the effect of class distribution on tree induction. Int. J. Artif. Intell. Res. 19, 315–354 (2003)
Wosiak, A., Zakrzewska, D.: Integrating correlation-based feature selection and clustering for improved cardiovascular disease diagnosis. Complexity 2018, 1–11 (2018)
Wu, X.: Research on the influence of crossover probability and mutation probability in ga-svm model. In: 9th IEEE Int. Conf. on Electronics Information and Emergency Communication (ICEIEC), pp. 1–7 (2019)
Yan, B., Han, G., Sun, M., et al A novel region adaptive smote algorithm for intrusion detection on imbalanced problem. In: 3rd IEEE Int. Conf. on Computer and Communications (ICCC), pp. 1281–1286 (2017)
Yang, S.: Genetic algorithms with elitism-based immigrants for changing optimization problems. In: Workshops on Applications of Evolutionary Computation, pp. 627–636 (2007)
Yeshalem, G.D., Chen, H., Mohi Yu Din, B.: Bootstrap-based homogeneous ensemble feature selection for network intrusion detection system. In: Developments of Artificial Intelligence Technologies in Computation and Robotics: Proc. 14th Int. FLINS Conf. (FLINS 2020), pp. 27–34 (2020)
Zhang, X., Ran, J., Mi, J.: An intrusion detection system based on convolutional neural network for imbalanced network traffic. In: 7th IEEE Int. Conf. on Computer Science and Network Technology (ICCSNT), pp. 456–460 (2019)
Zhang, Y., Zhang, H., Zhang, X., et al.: Deep learning intrusion detection model based on optimized imbalanced network data. In: 18th IEEE Int. Conf. Communication Technology (ICCT), pp. 1128–1132 (2018)
Zhu, T., Lin, Y., Liu, Y.: Synthetic minority oversampling technique for multiclass imbalance problems. Pattern Recognit. 72, 327–340 (2017)
Acknowledgements
We would like to thank the government of China and Southwest Jiaotong University for providing us the necessary facilities. We would also like to thank the editors and reviewers for their valuable comments.
Author information
Authors and Affiliations
Contributions
YG conceived the idea and developed the main framework. He also implemented all the experiments and was in charge of writing. HC checked and revised the correctness of the theory and the writing. Both the authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Damtew, Y.G., Chen, H. SMMO-CoFS: Synthetic Multi-minority Oversampling with Collaborative Feature Selection for Network Intrusion Detection System. Int J Comput Intell Syst 16, 12 (2023). https://doi.org/10.1007/s44196-022-00171-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s44196-022-00171-9