Home
Projects
Tools
Articles/Thoughts
Related Links
Subscribe to Blog RSS feed
RSS | Atom | E-mail

org.jwall.web.audit

This is a small web-audit library that is related to reading and writing audit-log data. Right now it includes a basic java parser to read ModSecurity audit-logs (version 1.x and 2.x), even in concurrent log-format (ModSecurity 2.x).

The library is part of a project for automatically learning rulesets for a web-application in the sense of a positive security-model (aka white-listing approach). However this part of the project is not stable enough to be published (there is a demo version available, contact me via mail if you're interested).

Features

The current version of the web-audit library supports reading various file-types, including concurrent audit-logs. Reading can either be done in a active way from within an application or in a separate thread which listens for events to come and dispatches them to various registered listeners. This is provided by a small and simple listener-framework which to make it easy to write tools for handling audit-event data.

Download

The library is compiled for Java 1.5 and thus only usable with Java 1.5+. A jar-archive of a built of the current release (version 0.2.15) is just 148k in size:

Changes in 0.2.15

  • Fixes smore more parser issues, which resulted in events being skipped during the parsing process
  • The Collector did not work properly on a Windows machine, now fixed.
  • Added support for events being sent to an ssl-enabled ModSecurity Console.

Changes in 0.2.13

This release fixes a small bug within the parser component, regarding missing/empty sections and requests using the CONNECT method.

Accessing the Source

Addiditonally there is also the source code available: org.jwall.web.audit-0.2.13-src.zip
If you are interested in contributing or submitting a patch, you may perhaps want to access the sources via subversion. Read-only subversion is available at

Documentation

Though the library is already usable, it currently lacks a comprehensive documentation. There is an Overview-page with an overview of the library's intended use and a few code-examples. For more detailed information, there is currently only the JavaDoc pages:

Java-based Collector 

The ModSecurity Console provides a nice view on the relevant audit-events which are collected from various webservers running ModSecurity. These servers send their events to the Console using a small perl-skript or the native mlogc-application.

Using the web-audit library I developed a Java-based collector. It runs outside the Apache webserver and listens for either concurrent or serial audit-data. This way you can also send audit-events to the console that are created by tools other than ModSecurity itself (like the WebTap for example). The Collector is included in the jar-archive of the web-audit library.

Using the current version of the web-audit library the Collector can be started by issuing 

   java -cp org.jwall.web.audit-0.2.9.jar org.jwall.Collector <config-file>
where config-file is the name of a simple properties file. This properties file contains the config like username, address of the ModSecurity Console and the audit-file to observe. To send all audit-events which written to the serial log-file /var/www/audit.log to the Console running on the local server this file looks like 
   org.modsecurity.console.host=localhost
   org.modsecurity.console.port=8886
   org.modsecurity.console.user=test
   org.modsecurity.console.password=sensor
   org.modsecurity.collector.serial-log=/var/www/audit.log
For reading the audit-events from a concurrent audit-log you need to specify the directory where the data files are written to (...collector.concurrent-log) as well as the location of the index-file (...collector.concurrent-index). 
   org.modsecurity.console.host=localhost
   org.modsecurity.console.port=8886
   org.modsecurity.console.user=test
   org.modsecurity.console.password=sensor
   org.modsecurity.collector.concurrent-log=/var/www/audit/
   org.modsecurity.collector.concurrent-index=/var/www/audit/index

AuditServer

The AuditServer is a small server application that allows for a continous monitoring of a logfile using a remote client. The server is started for a log-file and subsequently follows that file. Clients can connect to the server using an SSL socket. After a client successfully connected and authenticates itself properly, it will receive all events that are appended to the logfile that is being supervised.

To start the server on a specific serial audit-log file, simply issue the following command: 

   java -cp org.jwall.web.audit-0.2.9.jar org.jwall.AuditServer /path/to/audit.log
This will start the server on tcp-port 10001 and allow the user "admin" to connect using the password "secret". The format of the log-file (ModSecurity 1.x or 2.x) will be determined automatically by the server. Currently only serial logs are supported (support for concurrent logs will be added in a few days/weeks).
In order to specify a different user/password you can provide a text-file containing lines like user=password to the server (all within in one line): 
   java -cp org.jwall.web.audit-0.2.9.jar org.jwall.AuditServer \
       /path/to/audit.log --users user.txt
In this case the user-names are specified within the file users.txt. The log-file is assumed to be the first argument after the class name, i.e. org.jwall.AuditServer.

License

So far the library is put under the GNU Public License (GPL). If you're interested in using the library within a closed-source environment then feel free to contact me a chris (at) jwall.org.

Though the library is closely related to the ModSecurity module, the software itself is neither maintained nor otherwise affiliated with ModSecurity, Thinking Stone or Breach Security. It is as such not a product or official implementation of the aforementioned companies.