I was wondering if there is any tool that can parse pcap data and convert it to a csv file with the following information:
timestamp, bytes, uplink/downlink, some extra info..
Basically, the uplink/downlink could be seen by the IP/MAC address, and the extra info is not really needed, but what I mean with that is choose a specific field of a packet for example.
I have been trying some tools but I have not found the suitable one yet. Otherwise I will write a small parser. Thanks in advance!
TShark
Here are some examples:
$ tshark -r test.pcap -T fields -e frame.number -e eth.src -e eth.dst -e ip.src -e ip.dst -e frame.len > test1.csv $ tshark -r test.pcap -T fields -e frame.number -e eth.src -e eth.dst -e ip.src -e ip.dst -e frame.len -E header=y -E separator=, > test2.csv $ tshark -r test.pcap -R "frame.number>40" -T fields -e frame.number -e frame.time -e frame.time_delta -e frame.time_delta_displayed -e frame.time_relative -E header=y > test3.csv $ tshark -r test.pcap -R "wlan.fc.type_subtype == 0x08" -T fields -e frame.number -e wlan.sa -e wlan.bssid > test4.csv $ tshark -r test.pcap -R "ip.addr==192.168.1.6 && tcp.port==1696 && ip.addr==67.212.143.22 && tcp.port==80" -T fields -e frame.number -e tcp.analysis.ack_rtt -E header=y > test5.csv $ tshark -r test.pcap -T fields -e frame.number -e tcp.analysis.ack_rtt -E header=y > test6.csv
Look no further, wireshark is your best friend. It can open your pcap file and allow you to specify extra columns which you want. After this you can simply export them as csv. On the main interface, simply right on any one of the columns and select "column preference". This opens a new window which is very intuitive. Just add a new column and specify the field name. As simple as that.
I had tried tshark but trust me it becomes a bit annoying especially with this:
tshark: Read filters were specified both with "-R" and with additional command-line arguments."
This message pops up if you include too many columns or for whatever unknown reason.
It looks like you want Bro's connection logs:
bro -r trace.pcap
head conn.log
Output:
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path conn
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes
#types time string addr port addr port enum string intervacount count string bool count string count count count count
1258531221.486539 gvuu4KIHDph 192.168.1.102 68 192.168.1.1 67 udp - 0.163820 301 300 SF - 0 Dd 1 329 1 328
1258531680.237254 6nWmFGj6kWg 192.168.1.103 137 192.168.1.255 137 udp dns 3.780125 350 0 S0 - 0 546 0 0
1258531693.816224 y2lMKyrnnO6 192.168.1.102 137 192.168.1.255 137 udp dns 3.748647 350 0 S0 - 0 546 0 0
Now parse the relevant fields:
bro-cut ts id.orig_h id.orig_p id.resp_h id.resp_p service orig_bytes resp_bytes < conn.log | head
1258531221.486539 192.168.1.102 68 192.168.1.1 67 - 301 300
1258531680.237254 192.168.1.103 137 192.168.1.255 137 dns 350 0
1258531693.816224 192.168.1.102 137 192.168.1.255 137 dns 350 0
1258531635.800933 192.168.1.103 138 192.168.1.255 138 - 560 0
1258531693.825212 192.168.1.102 138 192.168.1.255 138 - 348 0
1258531803.872834 192.168.1.104 137 192.168.1.255 137 dns 350 0
1258531747.077012 192.168.1.104 138 192.168.1.255 138 - 549 0
1258531924.321413 192.168.1.103 68 192.168.1.1 67 - 303 300
1258531939.613071 192.168.1.102 138 192.168.1.255 138 - - -
1258532046.693816 192.168.1.104 68 192.168.1.1 67 - 311 300
Here is the python tool to divide the pcap into flows and output the extracted features into a CSV file
Try using flows_to_weka tool in python
This requires a version of scapy installed in your system and better to copy the scapy folder inside the weka folder. And copy the wfe.py, tcp_stream.py and entropy.py files inside the scapy folder. After you done this Your current directory should look something like this:
C:\Users\INKAKA\flows_to_weka\scapy
and copy the .pcap file into this folder and try running this command :
$python wfe.py -i input.pcap -t csv > output.csv
and you can also retrieve the features that you want by adding the required features in tcp_stream.py and wfe.py.
For reference you can visit : https://github.com/fichtner/flows_to_weka
You can do this from the Wireshark application itself:
File>Save) (if you have just
done a capture) File>Export Packet Dissesctions>as "CSV" [etc] Voila
Is it possible that we can set fields separator other than comma ? Because in my PCap file, if i set the separator=, then my data in output file (.csv) doesn't looks good because i have , in my most of the columns.
So i want to know that is there any way we can set the field separator like other charactors i.e., | (pip) etc
Thanks
sudo apt-get install argus-client
argus -r filename.pcap -w filename.argus
-r <FILE> Read FILE
-w <FILE> Write FILE
ra -r filename.argus -u -s <features-comma-seprated>
Example:
ra -r filename.argus -u -s rank, stime, ltime, dur
-r <FILE> Read FILE
-u Print time values using Unix time format (seconds from the Epoch).
-s Specify the fields to print.
The list of available fields to print can be found here
This information is copied from my original blog which you can read here
As noted in the comments to the question, to output the ip addresses for frames in a capture file in csv format use something like:
tshark -r <filename> -t fields -e ip.addr
See the tshark help for more information about options to set the separator and quoting characters in the csv output.
Field names can be determined by using Wireshark to examine the capture file and selecting a particular field in the details pane. The field name will be then shown in the status line at the bottom of the Wireshark window.
