Trident is a novel framework to detect unknown attack traffic in a fine-grained and incremental manner. It can realize known class classification, fine-grained unknown class detection, and incremental model updates (including sample increments and class increments). At a high level, Trident consists of three tightly coupled components named tSieve, tScissors, and tMagnifier. They are responsible for profiling traffic, determining outlier thresholds, and clustering respectively.
pip install scipy
pip install numpy
pip install pandas
pip install matplotlib
pip install sklearn
pip install torchThe AutoEncoder model architecture is stored in ./code/autoencoder.py.
autoencoder.pyThe Extreme Value Theory (EVT) calculations is stored in ./code/evt.py.
evt.pyA demo dataset based on KDDCup 99 is stored in ./code/data_loader.py.
data_loader.pyTo perform this test code, enter the ./code/ folder, run with
python main_process.pyFor sequence-based models, user can configure recurrent neural networks, could refer to I2RNN and Seq2Seq architectures. For the graph-based model, Graph U-Nets could be used to construct the reconstruction loss of graph data. Particularly, RNN and GNN models require corresponding feature processing.
Note: the scenario for this sample code is that one emerging class appears in the data stream each time. When considering multiple new classes appearing simultaneously, further clustering is only required in the buffer pool (corresponding to res_ls in main_process.py). For the ensemble of multiple clusters, could refer to FARE.
-
New Directions in Automated Traffic Analysis, Jordan Holland, Paul Schmitt, Nick Feamster, Prateek Mittal - CCS 2021
-
FlowPic: Encrypted Internet Traffic Classification is as Easy as Image Recognition, Tal Shapira, Yuval Shavitt - INFOCOM Workshops 2019
-
FlowLens: Enabling Efficient Flow Classification for ML-based Network Security Applications, Diogo Barradas, Nuno Santos, Luís Rodrigues, Salvatore Signorello, Fernando M. V. Ramos, André Madeira - NDSS 2021
-
ERNN: Error-Resilient RNN for Encrypted Traffic Detection towards Network-Induced Phenomena, Ziming Zhao, Zhaoxuan Li, Jialun Jiang, Fengyuan Yu, Fan Zhang, Congyuan Xu, Xinjie Zhao, Rui Zhang, Shize Guo - TDSC 2023
-
ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification, Xinjie Lin, Gang Xiong, Gaopeng Gou, Zhen Li, Junzheng Shi, Jing Yu - WWW 2022
-
Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection, Yisroel Mirsky, Tomer Doitshman, Yuval Elovici, Asaf Shabtai - NDSS 2018
-
Realtime Robust Malicious Traffic Detection via Frequency Domain Analysis, Chuanpu Fu, Qi Li, Meng Shen, Ke Xu - CCS 2021
-
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning, Min Du, Feifei Li, Guineng Zheng, Vivek Srikumar - CCS 2021
-
Detecting Unknown Encrypted Malicious Traffic in Real Time via Flow Interaction Graph Analysis, Chuanpu Fu, Qi Li, Ke Xu - NDSS 2023
-
Random Partitioning Forest for Point-Wise and Collective Anomaly Detection—Application to Network Intrusion Detection, Pierre-Francois Marteau - TIFS 2021
-
FlowPrint: Semi-Supervised Mobile-App Fingerprinting on Encrypted Network Traffic, Thijs van Ede, Riccardo Bortolameotti, Andrea Continella, Jingjing Ren, Daniel J. Dubois, Martina Lindorfer, David Choffnes, Maarten van Steen, Andreas Peter - NDSS 2020
-
Conditional Variational Auto-Encoder and Extreme Value Theory Aided Two-Stage Learning Approach for Intelligent Fine-Grained Known/Unknown Intrusion Detection, Jian Yang, Xiang Chen, Shuangwu Chen, Xiaofeng Jiang, Xiaobin Tan - TIFS 2021
-
Classification Under Streaming Emerging New Classes: A Solution Using Completely-Random Trees, Xin Mu, Kai Ming Ting, Zhi-Hua Zhou - TKDE 2017
-
FARE: Enabling Fine-grained Attack Categorization under Low-quality Labeled Data, Junjie Liang, Wenbo Guo, Tongbo Luo, Vasant Honavar, Gang Wang, Xinyu Xing - NDSS 2021