Engine on Inliniac

OISF engine development update(2)

Wed, 30 Sep 2009 18:30:37 +0000

Another quick update on the development of the OISF engine. Overall development is going great. Basics like signature keywords, stream reassembly, ip defragmentation are nearing completion. Unified1 + barnyard was already working for quite some time, but now we also have unified2 compatible output. I’ve tested this to work with barnyard2 and Sguil which works nicely.

We have the first versions of our new YAML based configuration format checked in, a brand new logging API, midstream pickup support in our Stream engine, native PFRING support and many other additions.

OISF engine development update

Sun, 16 Aug 2009 14:17:32 +0000

The last month has been crazy busy. Development of the engine is progressing nicely. My own role has been assigning tasks to our coders, guiding them, reviewing their work, integrating it and of course write code. We currently have nine people coding, not all full time though, and are still looking for more coders.

Progress has been made on a number of things: we have many more decoders, threading updates, a stats subsystem, stream tracking and reassembly, a L7 protocol parser framework and many more unittests. We’re working on OpenCL hardware accelaration, although we’re running into driver issues, so that may take some time before it’s usable.

OISF engine prototype: streams handling

Tue, 31 Mar 2009 15:36:27 +0000

I’ve been thinking about how to deal with streams in the OISF engine. We need to do stream reassembly to be able to handle spliced sessions, otherwise it would be very easy to evade detection. Snort traditionally used an approach of inspecting the packets individually and reassembling (part of) the stream in a pseudo packet, that was inspected mostly like a normal packet. Recent Snort versions, especially when Stream5 was introduced, have a so called stream api. This enables detection modules to control the reassembly better.

OISF engine prototype: threading

Sat, 28 Feb 2009 20:38:28 +0000

In Januari I first wrote about my prototype code for the OISF engine. The first thing I started with when creating the code was the threading. The current code can run as a single thread or with many threads. In my normal testing I run with about 11 threads, 10 of which handle packets, 1 is a management thread.

The basic principle in the threading is that a packet is always handled by one thread at a time only. The reason for this is that it saves a lot of locking issues. If there is more than one thread, the engine can handle multiple packete simultaniously.

OISF IDS/IPS engine prototype intro

Wed, 07 Jan 2009 11:24:07 +0000

For over a year I’ve been working on a prototype implementation of a new IDS/IPS engine for the Open Infosec Foundation. This is not necessarily going to be the engine we’ll be using in OISF, although it’s likely that at least some of the code will be used. Discussions about features for the engine are still ongoing ( wiki, list), once that settles down we’ll see whats usable and whats not. In the worst case I still think many parts like hashing functions, pattern matcher implementations, protocol decoders, etc can be used.