Suricata on Myricom capture cards
Myricom and OISF just announced that Myricom joined to OISF consortium to support the development of Suricata. The good folks at Myricom already sent me one of their cards earlier. In this post I’ll describe how you can use these cards already, even though Suricata doesn’t have native Myricom support yet. So in this guide I’ll describe using the Myricom libpcap support.
Getting started
I’m going to assume you installed the card properly, installed the Sniffer driver and made sure that all works. Make sure that in your dmesg you see that the card is in sniffer mode:
[ 2102.860241] myri_snf INFO: eth4: Link0 is UP [ 2101.341965] myri_snf INFO: eth5: Link0 is UP
I have installed the Myricom runtime and libraries in /opt/snf
Compile Suricata against Myricom’s libpcap:
./configure --with-libpcap-includes=/opt/snf/include/ --with-libpcap-libraries=/opt/snf/lib/ --prefix=/usr --sysconfdir=/etc --localstatedir=/var make sudo make install
Next, configure the amount of ringbuffers. I’m going to work with 8 here, as my quad core + hyper threading has 8 logical CPU’s.
pcap:
- interface: eth5
threads: 8
buffer-size: 512kb
checksum-checks: no
The 8 threads setting makes Suricata create 8 reader threads for eth5. The Myricom driver makes sure each of those is attached to it’s own ringbuffer.
Then start Suricata as follows:
SNF_NUM_RINGS=8 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
If you want 16 ringbuffers, update the “threads” variable in your yaml to 16 and start Suricata:
SNF_NUM_RINGS=16 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
It looks like you can use any number of ringbuffers, so not limited to a power of 2 for example.
Example with CPU affinity
You can also use Suricata’s built in CPU affinity settings to assign a worker to a cpu/core. In this example I’ll create 7 worker threads that will each run on their own logical CPU. The remaining CPU can then be used by the management threads, most importantly the flow manager.
max-pending-packets: 8192
detect-engine:
- sgh-mpm-context: full
mpm-algo: ac-bs
threading:
set-cpu-affinity: yes
cpu-affinity:
- management-cpu-set:
cpu: [ "0" ]
- detect-cpu-set:
cpu: [ "1-7" ]
mode: "exclusive"
prio:
default: "high"
pcap:
- interface: eth5
buffer-size: 512kb
threads: 7
checksum-checks: no
Then start Suricata with:
SNF_NUM_RINGS=7 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
This configuration will reserve cpu0 for the management threads and will assign a worker thread (and thus a ringbuffer) to cpu1 to cpu7. Note that I added a few more performance tricks to it. This config with 7 cpu pinned threads appears to be a little faster than the case where CPU affinity is not used.
Myricom has a nice traffic replay tool as well. This replays a pcap at 1Gbps:
snf_replay -r 1.0 -i 10 -p1 /path/to/pcap
Final remarks
The Myricom card already works nicely with Suricata. Because of the way their libpcap code works, we can already use the ringbuffers feature of the card. Myricom does also offer a native API. Later this year, together with Myricom, we’ll be looking into adding support for it.