Pyscanxtract

PyScanXtract.py

Description

pyScanXtract.py is a script, written in Python, that I wrote in the shape of forensicscontest (puzzle #4). It enables to analyse a pcap file and to produce statistics about potential portscans. Here is a list of available statistics:

• General information: pcap md5sum, number of packets, capture duration, ...
• Discovered hosts: list of identified hosts with mac addr. and vendors
• Scan types: List of detected scan types (TCP SYN, CONNECT, XMAS, UDP, ...)
• Hosts/targets: List of flows between attackers and targets
• IP options (utilisation of fragmentation)
• TCP and UDP ports status (open, closed, filtered, unfiltered)
• Detailed frames: show
• Distributions by frame/time of some parameters (sport, IP length, IPID, TTL, dport, seq, ack, ...)

Some screen captures are available here:

Architecture

pyScanXtract.py works with conjunction of PHP/MySQL. The following figure explains how it works:

[1] A capture is realized with a tool (e.g. tcpdump) capable of providing a pcap file. [2] pyScanXtract.py analyses provided pcap file. It looks for mac's vendor [3] and stores results in a MySQL database [4]. [5] It generates a html report. From this report, you can access to Web Interfaces (WI) [6] [7] [8] if you have Apache/PHP installed. [6] From the WI, you can access [6] Host IPID distribution by clicking on a host from the "Discovered Hosts" section (report.html). [7] The distribution analysis is also available by clicking on "Scan distrib. charact." from the "Scan types" section (report.html) [8] From report.html, if you click on a scan type (Scan types section) or on a port status (TCP and UDP ports analysis sections), the list of appropriate frames are displayed.

Installation

See/Download complete source code at Google code: http://pyscanxtract.googlecode.com/files/pyscan.tar.gz

If you don't plan to use the Web Interfaces, you just need a functional MySQL database, a valid oui.txt file (can be downloaded from IEEE Standards Association: http://standards.ieee.org/regauth/oui/oui.txt) and required Python libraries:

• optparse
• sys
• shutil
• pcapy
• impacket
• os
• MySQLdb
• datetime
• struct
• hashlib

Use provided pyscan.sql script to initialize your database. If needed, modify following variables from line 38 in pyScanXtract.py:

DBHOST = '127.0.0.1'    # Host to connect to
DBUSER = 'pyscan'       # User name to connect to database
DBPSWD = 'pyscan'       # Password to connect to database
DBNAME = 'pyscan'       # Database name

To use Web Interfaces, you will also have to modify line 42:

BASEWB = 'http://localhost/pyscan' # Base path for pyscan web interface
                                   # Don't put / at the end of the path
                                   # e.g. http://localhost/pyscan

If you have multiple websites on Apache, you can add a virtual directory in your httpd.conf (depending on your configuration, it is sometimes in extra/httpd-vhosts.conf). Refer to following URL for more information:

http://httpd.apache.org/docs/1.3/vhosts/examples.html

Usage

pyScanXtract.py can be called with following basic syntax:

$ ./pyScanXtract -r evidence04.pcap

Here are available options:

-h, --help
   Show help message and exit

-r <PCAP_FILE>, --read-file=<PCAP_FILE>
   Capture file to process (pcap format)

-o <OUTPUT_DIRECTORY>, --output=<OUTPUT_DIRECTORY>
   Reporting directory (default: ./report/) where report.html will be
   written

-f, --force
   Force overwriting of files. Use if an already existing report directory
   exists

-v <OUI_FILE>, --vendor-database=<OUI_FILE>
   Vendor database (default: ./oui.txt). This file can be downloaded from
   http://standards.ieee.org/regauth/oui/oui.txt.

-d, --dont-purge
   Don't purge existing data in the database. Default behavior is to first
   TRUNCATE pyscan table.

Once finished, pyScanXtract.py will produce a report in HTML format, available in report directory. Open it in your browser.

History and TODO

History

Version	Date	Description
1.1	2010-03-27	Window size added
1.0	2010-02-14	Initial rev.

TODO

• The script is based on a flow analysis. If many scans use the same combination of src:sport/dst:dport, they may not be detected.
• Window/Maimon scans are not detected by the script
• Script is quite slow.
• Open question: is there any way to make the difference between a TCP SYN scan and TCP Connect() when ports are all CLOSED?