This is a small web-audit library that is related to reading and writing audit-log data. Right now it includes a basic java parser to read ModSecurity audit-logs (version 1.x and 2.x), even in concurrent log-format (ModSecurity 2.x).
The library is part of a project for automatically learning rulesets for a web-application in the sense of a positive security-model (aka white-listing approach). However this part of the project is not stable enough to be published (there is a demo version available, contact me via mail if you're interested).
The current version of the web-audit library supports reading various file-types, including concurrent audit-logs. Reading can either be done in a active way from within an application or in a separate thread which listens for events to come and dispatches them to various registered listeners. This is provided by a small and simple listener-framework which to make it easy to write tools for handling audit-event data.
- Read and write serial audit files
- Read and write concurrent audit-logs
- Collector: send audit-events to the ModSecurity Console
- AuditServer: get a "live" view on an audit-log for easy debugging (see Debugging Rulesets)
- Offers a Java API for handling AuditEvents
The library is compiled for Java 1.5 and thus only usable with Java 1.5+. A jar-archive of a built of the current release (version 0.2.15) is just 120k in size:
Changes in 0.2.15
- Fixes smore more parser issues, which resulted in events being skipped during the parsing process
- The Collector did not work properly on a Windows machine, now fixed.
- Added support for events being sent to an ssl-enabled ModSecurity Console.
- Now including Apache commons-codec library for Base64 encoding/decoding
Changes in 0.2.13
- Parser bug-fix regarding missing/empty sections and requests using the CONNECT method
Accessing the SourceThere are no source-archives available, but the source can be accessed using the read-only subversion repository at
In order to check-out and build the library you just need a recent subversion tool and the ant build-tool. To retrieve the latest sources simply use
svn co https://secure.jwall.org/svn/org.jwall.web.audit/trunk/ web-auditwhich will download the latest version of the web-audit library into a local folder called
After the check-out building is straight forward using the ant build-tool. Calling ant in the source-folder
cd web-audit ant dist
will result in a binary release of the checked-out source.
Though the library is already usable, it currently lacks a comprehensive documentation. There is an Overview-page with an overview of the library's intended use and a few code-examples. For more detailed information, there is currently only the JavaDoc pages:
The ModSecurity Console provides a nice view on the relevant audit-events which are collected from various webservers running ModSecurity. These servers send their events to the Console using a small perl-skript or the native mlogc-application.
Using the web-audit library I developed a Java-based collector. It runs outside the Apache webserver and listens for either concurrent or serial audit-data. This way you can also send audit-events to the console that are created by tools other than ModSecurity itself (like the WebTap for example). The Collector is included in the jar-archive of the web-audit library.
Using the current version of the web-audit library the Collector can be started by issuing
java -cp org.jwall.web.audit-0.2.9.jar org.jwall.Collector <config-file>
config-file is the name of a simple properties file. This properties file contains
the config like username, address of the ModSecurity Console and the audit-file to observe. To send
all audit-events which written to the serial log-file
/var/www/audit.log to the Console
running on the local server this file looks like
org.modsecurity.console.host=localhost org.modsecurity.console.port=8886 org.modsecurity.console.user=test org.modsecurity.console.password=sensor org.modsecurity.collector.serial-log=/var/www/audit.log
For reading the audit-events from a concurrent audit-log you need to specify the directory where
the data files are written to (
...collector.concurrent-log) as well as the location of
the index-file (
org.modsecurity.console.host=localhost org.modsecurity.console.port=8886 org.modsecurity.console.user=test org.modsecurity.console.password=sensor org.modsecurity.collector.concurrent-log=/var/www/audit/ org.modsecurity.collector.concurrent-index=/var/www/audit/index
The AuditServer is a small server application that allows for a continous monitoring of a logfile using a remote client. The server is started for a log-file and subsequently follows that file. Clients can connect to the server using an SSL socket. After a client successfully connected and authenticates itself properly, it will receive all events that are appended to the logfile that is being supervised.
To start the server on a specific serial audit-log file, simply issue the following command:
java -cp org.jwall.web.audit-0.2.9.jar org.jwall.AuditServer /path/to/audit.log
This will start the server on tcp-port 10001 and allow the user "admin" to connect using the password "secret". The format of the log-file (ModSecurity 1.x or 2.x) will be determined automatically by the server. Currently only serial logs are supported (support for concurrent logs will be added in a few days/weeks).
In order to specify a different user/password you can provide a text-file containing lines like
user=password to the
server (all within in one line):
java -cp org.jwall.web.audit-0.2.9.jar org.jwall.AuditServer \ /path/to/audit.log --users user.txt
In this case the user-names are specified within the file
users.txt. The log-file is assumed to be the first
argument after the class name, i.e.
So far the library is put under the GNU Public License (GPL). If you're interested in using the library within a closed-source environment then feel free to contact me a chris (at) jwall.org.
Though the library is closely related to the ModSecurity module, the software itself is neither maintained nor otherwise affiliated with ModSecurity, Thinking Stone or Breach Security. It is as such not a product or official implementation of the aforementioned companies.