Lightweight MITM Proxy - HoneyProxy

HoneyProxy is a lightweight man-in-the-middle proxy that helps you analyze HTTP(S) traffic flows. It is tailored to the needs of security researchers and allows both real-time and log analysis. Being compatible with mitmproxy, it focuses on features that are useful in a forensic context and allows extended visualization capabilites.

HoneyProxy is developed as a HTML5 browser-based application working on top of a logging core written in Python. It is primarily developed by Maximilian Hils and mentored by Guillaume Arcas as part of the Honeynet Google Summer of Code 2012 project.


Features

ºAnalyze HTTP(S) traffic on the fly
ºFilter and highlight traffic, regex support included.
ºReport Generation for saved flows, including a live JS editor.
ºSave HTTP conversations for later analysis
ºMake scripted changes with Python, e.g. remove Cache Header.
ºbased on and compatible to mitmproxy.
ºcross-platform (Windows, OSX and Linux)
ºSSL interception certs generated on the fly

Lightweight MITM Proxy: HoneyProxy


Quick start

Download the latest release, a development snapshot or clone the git repo:

git clone --recursive git://github.com/mhils/HoneyProxy.git.

Install all dependencies:

pip install pyOpenSSL pyasn1 Twisted Autobahn

Windows users: Install the binaries for pyOpenSSL and Twisted manually.
Ubuntu / Debian users: Install twisted as a package (sudo apt-get install python-twisted)

Start HoneyProxy with:

python honeyproxy.py

or

python honeyproxy.py --help.

If you don’t use a modern browser, a kitten will die. We currently support both Firefox and Chrome!

Most command line parameters are documented in the mitmproxy docs.


Dependencies

ºPython 2.7.x.
ºpyOpenSSL 0.12 or newer.
ºpyasn1 0.1.2 or newer.
ºTwisted 12.3.0 or newer.

Search

A major feature of HoneyProxy is the ability to filter requests based on different criteria. HoneyProxy has a rich set of modifiers to allow fine-grained search requests:

ºRegular Search: If you don’t supply any modifiers, HoneyProxy will show all flows that contain the given string. This is case-insensitive.
ºCase Sensitive Search: If your search request starts with an equal sign ( =filter ), search is performed case-sensitive.
ºInverse Search: If your search request starts with an exclamation mark ( !filter ), all requests that don’t match the criteria are displayed.
ºRegular Expressions: If your search request starts with a tidle sign ( ~param=(foo|bar|[\d]+) ), your input is treated as a regular expression. You can combine this with the inverse modifier (!~). Please note that regular expressions are always case-sensitive


Dump conversations into a directory structure


HoneyProxy can dump all response contents into a directory-like structure. For example if you request example.com/files/foo.zip, foo.zip will be placed in $dumpdir/example.com/files/foo.zip. However, there are some restrictions with the filesystem:

ºAs file and folder names are limited to different lengths on different operating systems, HoneyProxy is going to cut off long directory and/or file names ([...] in the folder name or file name is a good indicator for this).
ºThe content of a unique URL is not guaranteed to be identical when calling it twice. To handle this transparently, HoneyProxy creates a second file if the response contents don’t match.
ºAs example.com/foo/ can be both a resource and a directory, HoneyProxy appends [dir] to a directory if a resource with the same name exists. This might lead to the problem that example.com/foo/bar.zip and example.com/foo[dir]/baz.zip seem to be in the same directory.

Conclusion: Don’t assume that the --dump-dir option creates an exact representation of your HTTP requests. If you keep that in mind, it’s still a very powerful tool for visualization!


Multiuser Capability

Due to its client-server architecture, HoneyProxy is not restricted to a single GUI Session. You can open the HoneyProxy GUI on multiple machines and browse the same dump simultaneously. It also works in live mode, all client’s get a notice of new flows.


Usage Instructions


When connection from a remote machine, make sure to open both the GUI and the WebSocket port (8081 and 8082 by default). Feel free to configure them using the corresponding command line flags.

The HoneyProxy GUI is protected from unauthorized access via Basic Auth. To access the GUI, you need valid credentials (HoneyProxy adds them by default when opening the browser). While the username is constant, the password is usually a 32 digit random string. It is displayed on the command line after HoneyProxy has been started. If you are working in a trusted environment, you can change this by using the --api-auth command line flag.

Basic Auth User: honey
Basic Auth Pass: displayed on the command line or specified by --api-auth


Programmatic Access with JavaScript / API


While HoneyProxy has inherited a great Python API from mitmproxy (docs), it also provides access to flows via JavaScript. This can be extremly powerful for developers who are interested in extending HoneyProxy or JavaScript hackers who want to run custom queries on a set of flows. For short, HoneyProxy stores all flows in a Backbone Collection (window.HoneyProxy.traffic). A flow contains a request object, a response object, an error object and its unique id. For easified access, we have ES5 proxy objects for both request and response directly as attributes of the Flow (e.g. HoneyProxy.traffic.get(0).request.host). They are stateless wrappers with ES5 getters for the original model attributes.


HoneyProxy  mitmproxy


HoneyProxy is build on top of the excellent mitmproxy. It provides us with a solid proxy base and a great API. The features below are inherited by HoneyProxy and are part of the mitmproxy code base. We show them here, because they are highly useful and also part of HoneyProxy. Credit goes to mitmproxy though

Anticache

If you want to access request content of a cached file, specify the --anticache command line flag. It removes all caching headers from HTTP requests. This is a feature of mitmproxy and documented in the mitmproxy docs.

Transparent Mode

This is an currently undocumented feature of the latest mitmproxy trunk code, but it’s already in HoneyProxy! Just run HoneyProxy with the -T switch and add a proper iptables rule. This currently only works on Linux.

Replacements

This is a feature of mitmproxy and documented in the mitmproxy docs.

Setting up SSL interception
This is a feature of mitmproxy and documented in the mitmproxy docs. The default certificate directory for HoneyProxy is ~/mitmproxy/.

Programmatic Access with Python – mitmproxy Scripts

HoneyProxy has inherited a great Python API from mitmproxy (docs). You can find a lot of examples in the mitmproxy repo.

So much more…


If you find a command-line switch that is not documented here, head over to the mitmproxy website for details

Download HoneyProxy