Intrusion Detection/Prevention System (IDS/IPS) Testing Framework - pytbull

pytbull is an Intrusion Detection/Prevention System (IDS/IPS) Testing Framework for Snort, Suricata and any IDS/IPS that generates an alert file. It can be used to test the detection and blocking capabilities of an IDS/IPS, to compare IDS/IPS, to compare configuration modifications and to check/validate configurations.

The framework is shipped with about 300 tests grouped in 11 testing modules:

1. badTraffic: Non RFC compliant packets are sent to the server to test how packets are processed.
2. bruteForce: tests the ability of the server to track brute force attacks (e.g. FTP). Makes use of custom rules on Snort and Suricata.
3. clientSideAttacks: this module uses a reverse shell to provide the server with instructions to download remote malicious files. This module tests the ability of the IDS/IPS to protect against client-side attacks.
4. denialOfService: tests the ability of the IDS/IPS to protect against DoS attempts
5. evasionTechniques: various evasion techniques are used to check if the IDS/IPS can detect them.
6. fragmentedPackets: various fragmented payloads are sent to server to test its ability to recompose them and detect the attacks.
7. ipReputation: tests the ability of the server to detect traffic from/to low reputation servers.
8. normalUsage: Payloads that correspond to a normal usage.
9. pcapReplay: enables to replay pcap files
10. shellCodes: send various shellcodes to the server on port 21/tcp to test the ability of the server to detect/reject shellcodes.
11. testRules: basic rules testing. These attacks are supposed to be detected by the rules sets shipped with the IDS/IPS.

It is easily configurable and could integrate new modules in the future.

There are basically 5 types of tests:

1. socket: open a socket on a given port and send the payloads to the remote target on that port.
2. command: send command to the remote target with the subprocess.call() python function.
3. scapy: send special crafted payloads based on the Scapy syntax
4. client side attacks: use a reverse shell on the remote target and send commands to it to make them processed by the server (typically wget commands).
5. pcap replay: enables to replay traffic based on pcap files

Architecture

Remote mode

In this mode, the IDS is plugged on the span port (or port mirroring) of the core switch and is configured in promiscuous mode. The IDS analyzes all traffic that goes through the core switch. Malicious files can be downloaded either by pytbull or by the server. This mode is called "remote".

Local mode

In this mode, files are downloaded on the client pytbull is started from.

IDS mode with attacked server in DMZ

In this configuration, a firewall splits the network into 3 parts (lan, wan, dmz). The IDS is plugged in a span port (or port mirroring) of the switch with its interface configured in promiscuous mode. It will analyze every traffic that is sent to the LAN interface of the firewall.

IPS mode

In this configuration, a firewall splits the network into 3 parts (lan, wan, dmz). The IDS is plugged between pytbull and the firewall. To give the IDS a chance to detect the malicious files, pytbull has to download the infected files itself.

IPS mode with attacked server in DMZ

In this configuration, a firewall splits the network into 3 parts (lan, wan, dmz). The IDS is plugged between pytbull and the firewall. Malicious files have to be downloaded by pytbull directly to give the IDS a chance to detect them.

Usage

If you have selected the clientSideAttacks module (see configuration file section for more information), you will need to start the reverse shell on the server. Following command uses port 34567/tcp:

$ ./pytbull-server.py -p 34567

Since the files are downloaded in the current directory, you can create a pdf/ directory and start pytbull from the parent location:

$ mkdir pdf/
$ cd pdf/
$ ../pytbull-server.py -p 34567

Then start pytbull (on the client side). An example to start pytbull tests against 192.168.100.48, running Snort:

$ sudo ./pytbull -t 192.168.100.48

Notice that you will need to adapt (config.cfg) the port used by the reverse shell if you use the optional parameter -p on remote side.

Download pytbull