š” Update: Online application with anomaly detection capabilities is available here: https://www.h21lab.com/applications/logs-analyzer
Python scripts which can help to analyze anomalies in pcap file.
Source code is located at https://github.com/H21lab/Anomaly-Detection.
Scripts use as input json generated from pcap by the following command:
./tshark -T ek -x -r input.pcap > input.pcap.json
Unsupervised learning by using autoencoder neural network by using tensorflow.
Script using tensorflow classifier neural network and tshark input to detect anomalies. Neural network should be first trained by normal json pcap and anomaly json pcap with defined fields identifying relevant protocol fields.
Run script:
cat input.pcap.json | python ad_tf.py -i normal.pcap.json -a anomaly.pcap.json -f field_1 field_2 …. field_n
Simple script simulating SQL GROUP BY and ORDER BY in pcap. Input is only single pcap and defining the fields in pcap which should be used for grouping and ordering.
Run script:
cat input.pcap.json | python ad_simple.py field_1 field_2 …. field_n
UPDATES:
2.6.2019: Tensorflow IO project: issue on github has been opened to enable direct pcap data ingestion
19.6.2019: Tensorflow IO project: support to pcap dataset has been added