Sguil 0.8.0 was recently released, so it was time for an upgrade. Since I remembered the last major upgrade to be quite a bit of work I wasn’t looking forward to the new upgrade. However, to my surprise it was a breeze. Here is what I did.

On my Sguild server called “owl” – I’d like to think it’s very wise – I first went to my sguil directory, where the CVS checkout lives. There I did a “cvs up”. Next it was time to upgrade the database:

…

The last few years Vuurmuur development has been very slow, not to say pretty much stagnant. This had a couple of reasons. The first is that my attention was drawn to other projects, mostly Suricata these days. The second reason is that Vuurmuur pretty much does all I want. The third reason is that despite some minor contributions, no other developer has stepped up to take over.

Meanwhile, people continued using Vuurmuur, it made it’s way into Debian, got removed from it again, made it’s way into Ubuntu. Lately, every few weeks someone would ask me if Vuurmuur was still being developed. My answer always was “yes, but very slowly”.

…

January has been a productive month for Suricata, especially for the IPS part of it. I’ve quite some time on adding support to the stream engine to operate differently when running inline. This was needed as dropping attacks found in the reassembled stream or the application layer was not reliable. Up until now the stream engine would offer the reassembled stream to the detection engine as soon as it was ACK’d. This meant that by definition the packets containing the data had already passed the IPS device. Simply switching to sending un-ACK’d data to the detection engine would have it’s own set of issues.

…

Today exactly one year ago we released the first public version of Suricata, tagged 0.8.0. It was the first beta version. Six months later we released Suricata 1.0.0, the first stable release. Since then we’ve been doing 3 more releases: 1.0.1, 1.0.2 and 1.1 beta 1.

It has been an very exciting year, with a lot of press and community interest for our project. Also, a lot of work has been done in the past year. I already wrote that our performance has increased a lot.

…

A question I see quite often is, can I listen on multiple interfaces with a single Suricata instance? Until now the answer always was “no”. I’d suggest trying the “any”-pseudo interface (suricata -i any), with an bpf to limit the traffic or using multiple instances of Suricata. That last suggestion was especially painful, as one of the goals of Suricata is to allow a single process to process all packets using all available resources.

…

Today we’ve released Suricata 1.1 beta 1, the first beta of the upcoming Suricata 1.1 release. The official release announcement is here on the OISF website.

The main focus of the new release has been to improve performance and to add support to the features the new ET/ETpro ruleset needs. ET and ETpro have rulesets specially tuned and geared for Suricata. We’re still missing some new rule keywords that are used by VRT, so in the 1.1 beta 2 release we’ll address that.

…

The last months we’ve been working hard on improving Suricata. So hard actually, that we’ve drifted a bit from our original goal of doing a 1.0.3 “maintenance” release. Instead, the new release will be 1.1beta1. The change to 1.1 is to indicate the large number of changes, the beta1 is to … indicate the large number of changes :)

As you may know, Will Metcalf moved on to join Qualys. A significant loss to our project as Will was one of our founding members and is hard to replace in his role as QA lead. Not having a full time QA person on the team right now is a reason for us to decide we’re in need of a beta cycle for the next release.

…

’tcmalloc’ is a library Google created as part of the google-perftools suite for speeding up memory handling in a threaded program. It’s very simple to use and does work fine with Suricata. Don’t expect magic from it, but it should give you a few percent more speed.

On Ubuntu, install the libtcmalloc-minimal0 package:

apt-get install libtcmalloc-minimal0

Then run Suricata as follows (on a single line):

LD_PRELOAD="/usr/lib/libtcmalloc_minimal.so.0" ./src/suricata -c suricata.yaml -i eth0

…

The last weeks I’ve been spending quite a bit of time improving Suricata’s performance, making good progress. I did a lot of optimizations all over the code, but the most significant is a new way of prefiltering signatures for inspection. I’ll briefly explain the concept here.

But first a quick explanation of how Suricata selects signatures for inspection. When Suricata starts, it organizes signatures into groups, called SigGroupHead in the code. To reduce the number of signatures that need inspection for each packet, the grouping is done on quite a few properties: flow direction, protocol, src ip, dst ip, src port, dst port. Even though this grouping is quite aggressive, a single SigGroupHead can still contain many thousands of signatures. For example Emerging Threats web-client sigs will almost all end up in the same SigGroupHead.

…

After some well deserved vacation I’m getting back up to speed in Suricata development. Luckily most of our dev team continued to work in my absence, making today’s 1.0.2 release possible.

The main focus of this release was fixing the TCP stream engine. Judy Novak found a number of ways to evade detection. See her blog post describing the issues.

The biggest other change is the addition of a new application layer module. The SSH parser parses SSH sessions and stops detection/inspection of the stream after the encrypted part of the session has started. So this is mainly a module focused on reducing the number of packets that need inspection, just like the SSL and TLS modules.

…