The Suricata development training at RAID 2012 next week is going to happen, so please all RSVP. It’s free!

If you’re planning to attend, please let me know what topics you are interested in. We have core devs in the room, so we can go hardcore on everything from the threading to packet capture to CUDA to pattern matching… also more straightforward stuff like extending Suricata with new keywords, log modules, etc.

…

So Will started bugging me (again) on doing scripting from Suricata and I gave in. Just committed extremely immature, incomplete, experimental luajit scripting support.

What it does is that it adds a new keyword, “luajit”. There is one argument, a script name. That script is then loaded from your rules directory and ran against a packet. No flow, http or any of that right now, just packets.

Example rule: alert tcp any any -> any any (msg:"LUAJIT test"; luajit:test.lua; sid:1;)

…

I just released a new Vuurmuur version. The last release was in 2009, so it has been a while.

This release adds basic IPv6 support. The state of the IPv6 support is incomplete, but quite functional.

Supported features are:

- rules generation - log viewing - setting IPv6 addresses in hosts, networks and interfaces

Unsupported features are:

- connection viewer - NAT - blocklist - IPv6 address to Vuurmuur name conversion in the log

…

Since this morning Suricata 1.3.1 is available. The main focus of this release was fixing a number of bugs. See the list of closed bugs, the release notes and the upgrade instructions.

As a bonus, I applied a set of patches by Eric Leblond. Eric has been trying to push AF_PACKET to the limit and has achieved some spectacular results with it. Read all about his quest to get to 10Gbps here on Eric’s blog.

…

We’re considering to offer a Suricata development training day around the next OISF brainstorm meeting. That would be in Amsterdam around the RAID conference, in early September.

Topics we could cover:

- code/development overview - create/extend detect module - create/extend output module - app layer module - proto detection - …

The training would probably be free as it’s an excercise for us as well, so we’d just want honest feedback in return :)

…

Myricom and OISF just announced that Myricom joined to OISF consortium to support the development of Suricata. The good folks at Myricom already sent me one of their cards earlier. In this post I’ll describe how you can use these cards already, even though Suricata doesn’t have native Myricom support yet. So in this guide I’ll describe using the Myricom libpcap support.

Getting started

I’m going to assume you installed the card properly, installed the Sniffer driver and made sure that all works. Make sure that in your dmesg you see that the card is in sniffer mode:

…

One of the new features in Suricata 1.3 is a new content modifier called http_user_agent. This allows rule writers to match on the User-Agent header in HTTP requests more efficiently. The new keyword is documented in the OISF wiki. In this post, I’ll show it’s efficiency with two examples.

Example 1: rarely matching UA

Consider a signature where the match if on a part of the UA that is very rare, so not part of regular User Agents. In my example “abc”.

…

Today, almost half a year after the last “stable” release, we released Suricata 1.3. I think this release is a big step forward with regard to maturity of Suricata. Performance and scalability have been much improved, just like accuracy and stability.

The official announcement can be found on the OISF site

In the last 6 months a lot of code has been changed:

384 files changed, 44332 insertions(+), 18478 deletions(-)

…

For a few months Suricata has been able to calculate the MD5 checksum of files it sees in HTTP streams. Regardless of extraction to disk, the MD5 could be calculated and logged. Martin Holste created a set of very cool scripts to use the logged MD5 to look it up at VirusTotal and some other similar services. This is done outside of Suricata. One thing I have been wanting to try is matching against these MD5’s in Suricata itself.

…

For the Suricata 1.3beta1 release, one of our goals was to improve the scalability of the engine when running on many cores. As the graph below shows, we made a good deal of progress.

The blue line is an older 1.1 version, the yellow line is 1.3dev. It clearly shows that 1.1 peaked at 4 cores, then started to get serious contention issues. 1.3dev scales nicely beyond that, up to 24 cores in this test (four 6core AMD cpu’s). Tilera recently demonstrated Suricata on their many core systems, running a single Suricata process per cpu. Their cpu’s have 36 real cores.

…