Reading packet captures with Python and Scapy for quick offline analysis

Scapy in read-only mode

Scapy is a Python library for packet parsing and crafting. For defenders, its reading features are often enough: rdpcap or PcapReader loads packets, layers are accessed like dictionary keys, and fields are attributes. Working offline with captures you own avoids sending anything on the network.

Summaries

PcapReader streams packets so large files do not fill memory. The example counts source-destination pairs, lists DNS query names, and records TCP SYNs to find which ports were contacted.

Snippet

from collections import Counter
from scapy.all import PcapReader, IP, TCP, DNS, DNSQR

pairs, dns, syns = Counter(), Counter(), Counter()
with PcapReader('lab.pcap') as pr:
    for pkt in pr:
        if IP in pkt:
            pairs[(pkt[IP].src, pkt[IP].dst)] += 1
        if pkt.haslayer(DNSQR) and pkt[DNS].qr == 0:
            dns[pkt[DNSQR].qname.decode(errors='replace').rstrip('.')] += 1
        if TCP in pkt and pkt[TCP].flags == 'S':
            syns[(pkt[IP].dst, pkt[TCP].dport)] += 1

print('top talkers', pairs.most_common(5))
print('dns', dns.most_common(10))
print('syn targets', syns.most_common(10))

Lab: compare with tshark

Capture a few minutes of your own lab VM traffic while browsing, or use a sample capture. Run the Scapy script and then answer the same questions with tshark. Note the speed difference on a larger file. Then flag any host that sent SYNs to more than 50 distinct ports, a scan pattern you can test by running nmap against your own lab host during capture.

Snippet

tshark -r lab.pcap -q -z conv,ip | head -15
tshark -r lab.pcap -Y 'dns.flags.response == 0' -T fields -e dns.qry.name | sort | uniq -c | sort -rn | head
tshark -r lab.pcap -Y 'tcp.flags.syn==1 && tcp.flags.ack==0' -T fields -e ip.src -e tcp.dstport | sort -u | cut -f1 | uniq -c | sort -rn | head

Choosing a tool

Scapy is flexible and pure Python but slow on large captures. tshark is much faster and decodes far more protocols; pyshark wraps tshark for Python. dpkt is a fast, lightweight parser. For very large datasets, convert captures to Zeek logs and analyze those instead.

Diagram

Python packet tools

Pick for speed or flexibility.

Scapy
Flexible parsing; slower on big files.
pyshark
tshark dissectors from Python.
dpkt
Fast, lightweight parsing.
tshark
Fast command line with full dissectors.
Zeek logs
Best for very large captures.

Check yourself

1. Which Scapy class streams packets from a large file?

2. What does a TCP flags value of S without ACK indicate?

3. Which tool is usually faster on a multi-gigabyte capture?