Post

PCAP Auto Extractor

Network forensics is a critical component of cybersecurity investigations, enabling analysts to uncover malicious activities, understand attack patterns, and gather evidence.

Posted
Preview Image
By
9 min read
PCAP Auto Extractor

Mastering Network Forensics: A Deep Dive into PCAP Auto Extractor

The process of extracting and analyzing packet capture (PCAP) files can be time-consuming and complex, especially when dealing with large volumes of data. Today, I’m excited to introduce a powerful tool that simplifies this process: PCAP Auto Extractor.

What is PCAP Auto Extractor?

PCAP Auto Extractor is a robust, production-quality Bash script that automates the extraction of various protocol data from PCAP files using tshark (Wireshark’s command-line tool). It provides an organized approach to network forensics by:

  • Automatically extracting data from multiple protocols
  • Organizing outputs in a structured directory hierarchy
  • Providing an interactive menu system for targeted analysis
  • Integrating with specialized GitHub repositories for advanced analysis
  • Generating comprehensive summary reports

Getting Started

Installation and Prerequisites

Before using PCAP Auto Extractor, you’ll need to ensure you have the necessary dependencies installed. The script includes a handy requirements checker that makes this process straightforward:

1
2
3
4
5
6
7
8
9

# Clone the repository
git clone https://github.com/yourusername/pcap-auto-extractor.git
cd pcap-auto-extractor

# Make the scripts executable
chmod +x pcap-auto-extractor.sh requirements.sh

# Check and install dependencies
./requirements.sh --install

The script will automatically detect your operating system and install the required packages, including tshark, git, and python3, along with optional tools like exiftool and jq.

Basic Usage

Once you have the dependencies installed, using PCAP Auto Extractor is simple:

1

./pcap-auto-extractor.sh capture.pcap

This command will launch the interactive menu system, allowing you to choose how you want to analyze your PCAP file.

When you first run PCAP Auto Extractor, you’ll be greeted with a colorful, professional-looking menu:

1
2
3
4
5
6
7
8
9
10
11
12
13

╔══════════════════════════════════════════════════════════════╗
║                    PCAP Auto Extractor                        ║
║          Advanced Terminal Automation for PCAP Analysis        ║
╚══════════════════════════════════════════════════════════════╝

═══════════════════════════════════════════════════════════════════════
║                         PCAP Auto Extractor - Main Menu                  ║
═══════════════════════════════════════════════════════════════════════
1. Extract ALL (safe, modular)
2. Run a specific extractor
3. GitHub Tools
4. Show Summary
5. Exit

Let’s explore each option in detail.

Option 1: Extract ALL

The “Extract ALL” option is perfect for comprehensive analysis when you’re not sure what you’re looking for or want to perform a thorough investigation. When you select this option, the script will:

  1. Extract Metadata: Information about network interfaces, link types, and PCAP statistics.
  2. Analyze HTTP Traffic: Extract requests, hosts, POST bodies, authentication headers, cookies, and even exported objects like images or documents.
  3. Investigate DNS Activity: Extract queries, answers, TXT records, and all QNames for potential exfiltration indicators.
  4. Examine TCP Conversations: Identify SYN+ACK ports, TCP conversations, and top talkers.
  5. Uncover Kerberos Activity: Extract user/realm pairs, ciphers, and generate AS-REP candidates for password cracking.
  6. Collect Credentials: Find FTP credentials, SMTP subjects, and potential credentials in HTTP POST bodies.
  7. Detect TLS Handshakes: Extract JA3 fingerprints for TLS client fingerprinting.
  8. Perform Chained Analysis: Use initial findings to guide deeper analysis, such as extracting URIs for specific hosts or analyzing POST bodies for credentials.

After extraction is complete, you’ll be asked if you want to return to the menu or exit, giving you control over your workflow.

Option 2: Run a Specific Extractor

Sometimes you know exactly what you’re looking for, and running all extractors would be overkill. The “Run a specific extractor” option allows you to choose from a comprehensive list of targeted extractors:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

═══════════════════════════════════════════════════════════════════════
║                      Select an extractor to run:                       ║
═══════════════════════════════════════════════════════════════════════
  1. Meta - Interface listing
  2. Meta - Link types
  3. Meta - PCAP stats
  4. Meta - Kerberos fields
  5. HTTP - Requests
  6. HTTP - Hosts
  7. HTTP - POST bodies
  8. HTTP - Auth headers
  9. HTTP - Cookies
 10. HTTP - Export objects
 11. DNS - Queries
 12. DNS - Answers
 13. DNS - TXT records
 14. DNS - All QNames
 15. TCP - SYN+ACK ports
 16. TCP - Conversations
 17. TCP - IP conversations
 18. Kerberos - Users and realms
 19. Kerberos - Ciphers
 20. Kerberos - AS-REP candidates
 21. Credentials - FTP commands
 22. Credentials - FTP credentials
 23. Credentials - SMTP subjects
 24. TLS - JA3 fingerprints
 25. DNS - Exfiltration detection
 26. Chained - HTTP hosts analysis
 27. Chained - Kerberos analysis
 28. Chained - POST bodies analysis
 29. Back to main menu

Each extractor is color-coded by category (Meta, HTTP, DNS, TCP, Kerberos, Credentials, TLS), making it easy to find the specific analysis you need.

Option 3: GitHub Tools

PCAP Auto Extractor integrates with specialized GitHub repositories to extend its capabilities. This option allows you to clone and run tools for specific analysis tasks:

1
2
3
4
5
6
7

═══════════════════════════════════════════════════════════════════════
║                            GitHub Tools Menu                           ║
═══════════════════════════════════════════════════════════════════════
1. decrypt-winrm (https://github.com/h4sh5/decrypt-winrm)
2. ctf-tools (https://github.com/truongkma/ctf-tools)
3. john (https://github.com/openwall/john)
4. Back to main menu

For example, if you’ve extracted Kerberos AS-REP candidates, you can use the john or ctf-tools repository to run krbpa2john.py and prepare the data for password cracking. The script will guide you through providing any necessary inputs and save the results in your output directory.

Option 4: Show Summary

After performing extractions, you can generate a comprehensive summary of your findings:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

PCAP Auto Extractor - Analysis Summary
======================================
PCAP File: /home/user/capture.pcap
Analysis Date: Mon Jun 14 14:30:45 UTC 2023
Output Directory: /home/user/Documents/Pcap_Extracts/20230614_143045

Extracted Files:
----------------
http/: 5 files
  - http_requests.tsv
  - hosts.txt
  - post_bodies.txt
  - cookies.txt
  - uris_example.com.txt
dns/: 4 files
  - dns_queries.txt
  - dns_answers.tsv
  - txt_records.txt
  - exfil_suspicions.txt
kerberos/: 3 files
  - users_realms.tsv
  - ciphers_all.txt
  - asrep_candidates.txt
tcp/: 2 files
  - synack_srcports.txt
  - tcp_conversations.txt
creds/: 2 files
  - http_auth_headers.txt
  - ftp_user_pass.tsv
meta/: 4 files
  - interfaces.txt
  - linktypes.txt
  - io_phs.txt
  - fields_kerberos.txt
other/: 2 files
  - ftp_commands.tsv
  - smtp_subjects.txt
files/http/: 12 files

Intelligence Summary:
--------------------
HTTP: 8 unique hosts
HTTP: 24 POST requests
DNS: 156 queries
DNS: 3 exfiltration indicators
Kerberos: 5 user/realm pairs
Kerberos: 3 AS-REP candidates
TCP: Top conversations
               1       192.168.1.10    192.168.1.1      85     4285     4285      85
               2       192.168.1.10    8.8.8.8          42     1024     1024      42
               3       192.168.1.10    10.0.0.5         38      896      896      38
               4       192.168.1.10    192.168.1.20     32      768      768      32
               5       192.168.1.10    192.168.1.30     28      672      672      28
Credentials: 2 HTTP auth headers
Credentials: 1 FTP credentials
Files: 12 HTTP objects exported

If you use the --json-report flag, you’ll also get a machine-readable JSON summary for integration with other tools or automated workflows.

Advanced Usage

Command-Line Options

PCAP Auto Extractor supports several command-line options for advanced use cases:

1

./pcap-auto-extractor.sh capture.pcap --parallel 4 --json-report
  • --parallel N: Run up to N extractors in parallel (default: 1)
  • --dry-run: Show commands that would be executed without running them
  • --json-report: Generate a machine-readable JSON summary report
  • --tools-root DIR: Set the root directory for cloned tools (default: /tmp/tools)

Output Structure

The script creates a timestamped output directory in ~/Documents/Pcap_Extracts/ with a well-organized structure:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

~/Documents/Pcap_Extracts/YYYYMMDD_HHMMSS/
├── logs/
│   ├── run.log          # Main execution log
│   └── errors.log       # Error log
├── files/               # Exported binary files
│   ├── http/            # HTTP objects (images, documents, etc.)
│   ├── smb/             # SMB files
│   ├── nfs/             # NFS files
│   ├── tftp/            # TFTP files
│   └── ftp/             # FTP files
├── http/                # HTTP textual outputs
├── dns/                 # DNS outputs
├── kerberos/            # Kerberos outputs
├── tcp/                 # TCP outputs
├── creds/               # Credentials
├── meta/                # Metadata
├── other/               # Other protocols
├── tools/               # Cloned GitHub tools and outputs
├── SUMMARY.txt          # Human-readable summary
└── summary.json         # Machine-readable summary (if requested)

Real-World Use Cases

1. Investigating a Security Incident

Imagine you’re investigating a security incident and have a PCAP file from the affected network. You could:

  1. Run PCAP Auto Extractor with the “Extract ALL” option to get a comprehensive view of the network activity.
  2. Review the summary for any immediate red flags, such as unusual DNS queries or suspicious HTTP POST requests.
  3. Use the DNS exfiltration detection results to identify potential data exfiltration attempts.
  4. Examine extracted credentials to identify compromised accounts.
  5. If Kerberos activity is detected, use the GitHub Tools option to run krbpa2john and prepare AS-REP candidates for password cracking.

2. Malware Analysis

When analyzing malware communication:

  1. Use specific extractors to focus on HTTP and DNS traffic.
  2. Extract HTTP objects to retrieve any files downloaded by the malware.
  3. Analyze DNS queries to identify command and control servers.
  4. Use the TCP conversation analysis to understand network communication patterns.

3. Network Troubleshooting

For network troubleshooting scenarios:

  1. Extract TCP conversations to identify connection issues.
  2. Analyze HTTP requests and responses to identify application-level problems.
  3. Use the interface and link type information to verify network configuration.

Tips and Best Practices

  1. Start with Extract ALL: When analyzing a PCAP file for the first time, use the “Extract ALL” option to get a comprehensive view of the network activity.

  2. Review the Summary: The summary report provides a high-level overview of findings, making it easy to identify areas that need further investigation.

  3. Use Specific Extractors for Targeted Analysis: Once you’ve identified areas of interest, use specific extractors to dive deeper into those areas.

  4. Leverage GitHub Tools: Take advantage of the integrated GitHub tools to extend the capabilities of PCAP Auto Extractor for specialized analysis tasks.

  5. Keep Logs for Documentation: The script maintains detailed logs of all operations, which can be valuable for documentation and reporting purposes.

  6. Use Dry Run for Testing: Before running extractors on large PCAP files, use the --dry-run option to see what commands would be executed.

Conclusion

PCAP Auto Extractor streamlines the network forensics process by automating the extraction and analysis of packet capture data. Its modular design, comprehensive extraction capabilities, and integration with specialized tools make it a valuable addition to any cybersecurity professional’s toolkit.

Whether you’re investigating a security incident, analyzing malware, or troubleshooting network issues, PCAP Auto Extractor provides a structured and efficient approach to PCAP analysis. By automating the tedious aspects of data extraction and organization, it allows you to focus on the more important task of interpreting the results and uncovering insights.

project
pcap Linux Exploit
This post is licensed under CC BY 4.0 by the author.