pmacct and AS-Stats

by ohrensessel

AS-Stats is a handy tool to visualize traffic on a per-AS basis. We are making heavy use of it to spot optimization potential regarding traffic from or to our Freifunk Hamburg AS 49009.

In addition to the per-AS graphs showing ingress and egress traffic (separated for IPv4 and IPv6), a graph for each link showing the traffic caused by the top 10 AS (on that link) is generated.

Structure

We use pmacct as IPFIX probe (NetFlow might be a name you already heard) on the routers (which run bird as routing daemon) handling the traffic we want to visualize. For each network interface, a pmacctd instance is run.

Some other host within our AS runs AS-Stats, to which all pmacctd instances are reporting information regarding the traffic sampled by them.

Noteworthy, we let pmacctd do the aggreation per-AS, so that only a counter for egress respective ingress traffic per-AS is sent towards the AS-Stats instance, but no information regarding single flows.

For that pmacct needs to have information regarding the mapping from IP networks to ASs.

pmacct

Probably, it is a good idea to compile pmacct from source (not covered here...).

For each interface you want to monitor in AS-Stats, put a config file with the following content in /etc/pmacct/ (named e.g. pmacct.1.conf).

interface:<ifname>
nfprobe_source_ip:<ipOfPmacctHost>
daemonize:false
promisc:true
snaplen:200
pre_tag_map:/etc/pmacct/pre_tag.map
plugin_buffer_size:10240
plugin_pipe_size:10240000
sampling_rate:128
aggregate:etype,src_as,dst_as,vlan
nfacctd_as_new:file
networks_file:/var/cache/pmacct/netmap.txt
plugins: nfprobe[a]
nfprobe_version[a]:10
nfprobe_receiver[a]:<asstatsIp>:<asstatsPort>
nfprobe_direction[a]:tag
nfprobe_ifindex[a]:tag2

Replace <ifname> with the name of the interface to monitor and choose a source IP (<ipOfPmacctHost>) from which to send the IPFIX packets to AS-Stats, reachable at <asstatsIp>:<asstatsPort>.

The aggregate parameter defines that we want to distinguish traffic by EtherType (0x0800 for IPv4 and 0x86DD for IPv6), by source and destination AS, and by VLAN (if any). For an explanation of the other config parameters see the quite extensive pmacct wiki.

Note: We only sample each 128th packet here to reduce CPU load. Depending on the amount of traffic you are seeing you might want to adapt that value (but this has to be equal to the value configured in AS-Stats).

To map traffic to a certain interface for AS-Stats, pmacct has to add a tag (see parameter pre_tag_map above). We do this depending on the source respective destination mac address of the interfaces we are monitoring.

id=1 filter='ether dst <macInterface1>' jeq=nif1
id=2 filter='ether src <macInterface1>' jeq=nif1
id=1 filter='ether dst <macInterface2>' jeq=nif2
id=2 filter='ether src <macInterface2>' jeq=nif2
!
id2=70  label=nif1
id2=80  label=nif2

Replace the parameters <macInterfaceX> with the mac addresses of the interfaces you are monitoring. Each interfaces has a label (nif1 and nif2 in this case) which lead to a tag with a certain value (70 and 80 in this case).

Note: Due to laziness, we only have one pre_tag_map for all interfaces holding all the mac addresses.

AS-Stats

The corresponding configuration for AS-Stats could look like the following:

<ipOfPmacctHost> 70 link_1 Link 1 ABCDEF  128
<ipOfPmacctHost> 80 link_1 Link 2 FEDCBA  128

Mapping from IP networks to AS

pmacct needs information regarding the mapping of IP networks to AS. For that, the parameter networks_file gives the path to a file holding tuples of AS and network line by line.

We generate that file by the following, very ugly bash script.

#!/bin/bash

/usr/sbin/birdc sh route primary | gawk 'match($0, /AS([0-9]+)/, ary) \
{ print ary[1] "," $1 }' | sort -n > /tmp/.birdout
/usr/sbin/birdc6 sh route primary | gawk 'match($0, /AS([0-9]+)/, ary) \
{ print ary[1] "," $1 }' | sort -n > /tmp/.bird6out

cat /tmp/.birdout > /var/cache/pmacct/netmap.txt
cat /tmp/.bird6out >> /var/cache/pmacct/netmap.txt