Everything inline.
Video: Suricata 2.0 installation and quick setup
I’ve made a video on installing Suricata 2.0 on Debian Wheezy. The video does the installation, quick setup, ethtool config and shows a simple way to test the IDS.
It’s the first time I’ve made such a video. Feedback is welcome.
…Suricata 2.0 and beyond
Today I finally released Suricata 2.0. The 2.0 branch opened in December 2012. In the little over a year that it’s development lasted, we have closed 183 tickets. We made 1174 commits, with the following stats:
582 files changed, 94782 insertions(+), 63243 deletions(-)
So, a significant update! In total, 17 different people made commits. I’m really happy with how much code and features were contributed. When starting Suricata this was what I really hoped for, and it seems to be working!
…tcpreplay on Intel 82576
For my Suricata QA setup, I’m using tcpreplay on a dual port gigabit NIC. The idea is to blast out packets on one port and then have Suricata listen on the other part.
For the traffic replay I’m using tcpreplay 3.4.4 from the Ubuntu archive. As I have a lot of pcaps to process I intend to use the –topspeed option to keep runtimes as low as possible. This will result in approximately ~500Mbps on this box, as the pcaps come from a nas.
…Disabling Threading in Tcl8.5 in Debian
I’ve been spending the holidays to upgrade some of my own servers. One of them is the Sguil server I use. Until now it ran Debian Squeeze. On Debian Squeeze you could use tcl8.3, which has threading disabled. For Sguil tcl threading needs to be disabled:
ERROR: This version of tcl was compile with threading enabled. Sguil is NOT compatible with threading.
This is a compile time option in TCL, and the Debian Wheezy packages have it enabled by default. Here are the steps to create your own tcl deb with threading disabled:
…Suricata Development Update
With the holidays approaching and the 1.4.7 and 2.0beta2 releases out, I thought it was a good moment for some reflection on how development is going.
I feel things are going very well. It’s great to work with a group that approaches this project from different angles. OISF has budget have people work on overall features, quality and support. Next to that, our consortium supporters help develop the project: Tilera’s Ken Steele is working on the Tile hardware support, doing lots optimizations. Many of which benefit performance and overall quality for the whole project. Tom Decanio of Npulse is doing great work on the output side, unifying the outputs to be machine readable. Jason Ish of Emulex/Endace is helping out the configuration API, defrag, etc. Others, both from the larger community and our consortium, are helping as well.
…GPG key update
I have revoked my old gpg keys 0BA788C7 and F5DFF229. They are superseded by 926856B4. You can grab that from http://pgp.mit.edu.
Reason for the update is to get rid of the 1024 bit keys. I have no reason to believe they were compromised :)
…Suricata profiling per keyword
Last week I’ve added some more profiling options to Suricata. It’s part of the current git master. It’s enabled only when --enable-profiling and then through the suricata.yaml:
profiling:
# per keyword profiling
keywords:
enabled: yes
filename: keyword_perf.log
append: yes
This will output a table similar to below:
--------------------------------------------------------------------------
Date: 11/7/2013 -- 15:13:11
--------------------------------------------------------------------------
Stats for: total
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
threshold 355324491 190574 409 72276 1864.00 3625.00 1860.00
content 1274592063 534328 196738 312321 2385.00 2424.00 2362.00
pcre 56626031 11149 824 254562 5079.00 12234.00 4507.00
byte_test 153287955 128254 32109 67989 1195.00 1658.00 1040.00
byte_jump 3676404 2041 2041 15939 1801.00 1801.00 0.00
flow 38276182 22842 22842 63987 1675.00 1675.00 0.00
isdataat 580764 558 556 2427 1040.00 1040.00 1017.00
dsize 2212029 2062 2061 3711 1072.00 1072.00 789.00
flowbits 1677209 874 870 9873 1919.00 1923.00 884.00
itype 1653 2 1 1386 826.00 267.00 1386.00
icode 27383781 93827 2 25545 291.00 1021.00 291.00
flags 192751968 245519 189709 255639 785.00 753.00 892.00
urilen 6149297 6142 1099 28299 1001.00 1395.00 915.00
byte_extract 143091 78 78 7743 1834.00 1834.00 0.00
--------------------------------------------------------------------------
Stats for: packet
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
flow 38276182 22842 22842 63987 1675.00 1675.00 0.00
dsize 2212029 2062 2061 3711 1072.00 1072.00 789.00
flowbits 351171 294 290 5526 1194.00 1198.00 884.00
itype 1653 2 1 1386 826.00 267.00 1386.00
icode 27383781 93827 2 25545 291.00 1021.00 291.00
flags 192751968 245519 189709 255639 785.00 753.00 892.00
--------------------------------------------------------------------------
Stats for: packet/stream payload
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
content 1203990910 512902 183628 312321 2347.00 2365.00 2337.00
pcre 28087301 6598 54 254562 4256.00 12279.00 4190.00
byte_test 153287955 128254 32109 67989 1195.00 1658.00 1040.00
byte_jump 3676404 2041 2041 15939 1801.00 1801.00 0.00
isdataat 578172 556 554 2427 1039.00 1039.00 1017.00
byte_extract 143091 78 78 7743 1834.00 1834.00 0.00
--------------------------------------------------------------------------
Stats for: http uri
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
content 44775802 13102 8351 60993 3417.00 3257.00 3698.00
pcre 18284421 3646 97 61338 5014.00 8916.00 4908.00
isdataat 2592 2 2 1725 1296.00 1296.00 0.00
urilen 6149297 6142 1099 28299 1001.00 1395.00 915.00
--------------------------------------------------------------------------
Stats for: http raw uri
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
pcre 9534 2 0 4953 4767.00 0.00 4767.00
--------------------------------------------------------------------------
Stats for: http client body
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
content 1556904 441 181 58476 3530.00 2874.00 3986.00
pcre 63924 6 6 17358 10654.00 10654.00 0.00
--------------------------------------------------------------------------
Stats for: http headers
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
content 23688244 7631 4348 31098 3104.00 3311.00 2829.00
pcre 9998970 859 667 71904 11640.00 12727.00 7862.00
--------------------------------------------------------------------------
Stats for: http stat code
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
content 80052 39 20 3699 2052.00 2199.00 1898.00
--------------------------------------------------------------------------
Stats for: http method
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
content 476334 203 201 27240 2346.00 2351.00 1846.00
--------------------------------------------------------------------------
Stats for: http cookie
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
content 23817 10 9 2763 2381.00 2384.00 2358.00
pcre 181881 38 0 13095 4786.00 0.00 4786.00
--------------------------------------------------------------------------
Stats for: post-match
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
flowbits 1326038 580 580 9873 2286.00 2286.00 0.00
--------------------------------------------------------------------------
Stats for: threshold
--------------------------------------------------------------------------
Keyword Ticks Checks Matches Max Ticks Avg Avg Match Avg No Match
---------------- ----------- -------- -------- ----------- ----------- ----------- -----------
threshold 355324491 190574 409 72276 1864.00 3625.00 1860.00
The first part has the totals for all keywords. After this the stats are broken down per buffer type.
…Attending Hack.lu with the Suricata team
Next month I will be attending Hack.lu. The entire Suricata team will be present as well. We’ll be doing several meetings, including a training day on the 24th of October.
If you are close and interested in Suricata, please consider joining us. The training is free and does not require you to pay for the conference.
…Fixing "error: m4_defn: undefined macro: _m4_divert_diversion"
Ran into a problem with autotools today, thought I’d share my solution. First, the error only happened on an old system:
$ bash autogen.sh
Found libtoolize
Remember to add `AC_PROG_LIBTOOL' to `configure.ac'.
libtoolize: `config.guess' exists: use `--force' to overwrite
libtoolize: `config.sub' exists: use `--force' to overwrite
libtoolize: `ltmain.sh' exists: use `--force' to overwrite
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
configure.ac:1: error: m4_defn: undefined macro: _m4_divert_diversion
configure.ac:1: the top level
autom4te: /usr/bin/m4 failed with exit status: 1
aclocal: autom4te failed with exit status: 1
autoreconf: aclocal failed with exit status: 1
The autogen.sh file is this one: https://github.com/inliniac/suricata/blob/master/autogen.sh
…More on Suricata lua flowints
This morning I added flowint lua functions for incrementing and decrementing flowints. From the commit:
Add flowint lua functions for incrementing and decrementing flowints.
First use creates the var and inits to 0. So a call:
a = ScFlowintIncr(0)Results in a == 1.
If the var reached UINT_MAX (2^32), it’s not further incremented. If the var reaches 0 it’s not decremented further.
Calling ScFlowintDecr on a uninitialized var will init it to 0.
…