This guide walks through configuring SiLK from a source install on a Debian 6 host in order to collect NetFlow data from a Cisco router. The guides here and here written by CERT NetSA are quite good but lack some detail specific to the Debian distribution which required a bit of mucking about to get everything functioning correctly. This assumes that you have a Cisco router to send NetFlow data to a host on your network, in this case, a Debian host.
Installation:
First install a prerequisite.
$ sudo apt-get install libpcap-dev
Next untar and change into the SiLK directory. For Debian I found that using the /usr directory worked well. By default the configure script uses /usr/local in which it places the binaries, libraries, etc outside of Debians default paths.
$ ./configure --prefix=/usr --sysconfdir=/etc/silk --enable-data-rootdir=/netflow
--enable-ipv6 --enable-output-compression
Your output should be something along the following:
* Configured package: SiLK 2.4.5
* Host type: x86_64-unknown-linux-gnu
* Source files ($top_srcdir): .
* Install directory: /usr
* Root of packed data tree: /netflow
* Packing logic: via run-time plugin
* Timezone support: UTC
* Default compression method: SK_COMPMETHOD_ZLIB
* IPv6 support: YES
* IPFIX collection support: YES (-pthread -lfixbuf -lgthread-2.0 -lrt -lglib-2.0)
* Transport encryption support: NO (gnutls not found)
* IPA support: NO
* LIBPCAP support: YES (-lpcap)
* ADNS support: NO
* Python support: NO
* Build analysis tools: YES
* Build packing tools: YES
* Compiler (CC): gcc
* Compiler flags (CFLAGS): -I$(srcdir) -I$(top_builddir)/src/include -I$(top_srcdir)/src/include -DNDEBUG -O3 -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wformat=2 -Wdeclaration-after-statement -Wpointer-arith
* Linker flags (LDFLAGS):
* Libraries (LIBS): -lz -ldl -lm
Lastly:
$ make
$ sudo make install
Configuration:
Example files are available in the tarball that you extracted. Modified versions or notes for Debian and similar architectures available below.
/netflow/silk.conf in your data directory, the default is /data but I used /netflow as you can see in the configure toggle above. The changes I made were to reduce the number of sensors.
# The syntactic format of this file
# version 2 supports sensor descriptions, but otherwise identical to 1
version 2
sensor 0 s0 "Description for sensor S0"
sensor 1 s1
class all
sensors s0 s1
end class
# Editing above this line is sufficient for sensor definition.
/etc/silk/sensor.conf is the definition for the data coming in from your Cisco router:
probe s0 netflow-v5
listen-on-port 9990
protocol udp
accept-from-host 172.16.0.1
end probe
sensor s0
netflow-v5-probes s0
internal-ipblocks 172.16.0.0/24
external-ipblocks remainder
end sensor
/etc/silk/rwflowpack.conf:
### Packer configuration file -*- sh -*-
##
## The canonical pathname for this file is /usr/local/etc/rwflowpack.conf
##
## RCSIDENT("$SiLK: rwflowpack.conf.in 16306 2010-09-15 18:14:41Z mthomas $")
##
## This is a /bin/sh file that gets loaded by the init.d/rwflowpack
## wrapper script, and this file must follow /bin/sh syntax rules.
# Set to non-empty value to enable rwflowpack
ENABLED=yes
# These are convenience variables for setting other values in this
# configuration file; their use is not required.
statedirectory=/var/lib/rwflowpack
# If CREATE_DIRECTORIES is set to "yes", the directories named in this
# file will be created automatically if they do not already exist
CREATE_DIRECTORIES=yes
# Full path of the directory containing the "rwflowpack" program
BIN_DIR=/usr/sbin
# The full path to the sensor configuration file. Used by
# --sensor-configuration. YOU MUST PROVIDE THIS (the value is ignored
# when INPUT_MODE is "respool").
SENSOR_CONFIG=/etc/silk/sensor.conf
# The full path to the root of the tree under which the packed SiLK
# Flow files will be written. Used by --root-directory.
DATA_ROOTDIR=/netflow
# The full path to the site configuration file. Used by
# --site-config-file. If not set, defaults to silk.conf in the
# ${DATA_ROOTDIR}.
SITE_CONFIG=/netflow/silk.conf
# Specify the path to the packing-logic plug-in that rwflowpack should
# load and use. The plug-in provides functions that determine into
# which class and type each flow record will be categorized and the
# format of the files that rwflowpack will write. When SiLK has been
# configured with hard-coded packing logic (i.e., when
# --enable-packing-logic was specified to the configure script), this
# value should be empty. A default value for this switch may be
# specified in the ${SITE_CONFIG} site configuration file. This value
# is ignored when INPUT_MODE is "respool".
PACKING_LOGIC=
# Data input mode. Valid values are:
# * "stream" mode to read from the network or from probes that have
# poll-directories
# * "fcfiles" to process flowcap files on the local disk
# * "respool" to process SiLK flow files maintaining the sensor and
# class/type values that already exist on those records.
INPUT_MODE=stream
# Directory in which to look for incoming flowcap files in "fcfiles"
# mode or for incoming SiLK files in "respool" mode
INCOMING_DIR=${statedirectory}/incoming
# Directory to move input files to after successful processing. When
# in "stream" mode, these are the files passed to any probe with a
# poll-directory directive. When in "fcfiles" mode, these are the
# flowcap files. When in "respool" mode, these are the SiLK Flow
# files. If not set, the input files are not archived but are deleted
# instead.
ARCHIVE_DIR=${statedirectory}/archive
# When using the ARCHIVE_DIR, normally files are stored in
# subdirectories of the ARCHIVE_DIR. If this variable's value is 1,
# files are stored in ARCHIVE_DIR itself, not in subdirectories of it.
FLAT_ARCHIVE=0
# Directory to move an input file into if there is a problem opening
# the file. If this value is not set, rwflowpack will exit when it
# encounters a problem file. When in "fcfiles" mode, these are the
# flowcap files. When in "stream" mode, these are the files passed to
# any probe with a poll-directory directive.
ERROR_DIR= #${statedirectory}/error
# Data output mode. Valid values are "local" and "remote". "local"
# writes the hourly data files to the local disk. "remote" creates
# small files (called incremental files) that must be processed by
# rwflowappend to create the hourly files.
OUTPUT_MODE=local
# Directory in which the incremental files are written when the
# OUTPUT_MODE is "remote". Typically there is an rwsender deamon that
# polls this directory for new incremental files.
SENDER_DIR=${statedirectory}/sender-incoming
# Temporary directory in which to build incremental files prior to
# handing them to rwsender. Used only when OUTPUT_MODE is "remote".
INCREMENTAL_DIR=${statedirectory}/incremental
# The type of compression to use for packed files. Left empty, the
# value chosen at compilation time will be used. Valid values are
# "best" and "none". Other values are system-specific (the available
# values are listed in the description of the --compression-method
# switch in the output of rwflowpack --help).
COMPRESSION_TYPE=best
# Interval between attempts to check the INCOMING_DIR or
# poll-directory probe entries for new files, in seconds. This may be
# left blank, and will default to 15.
POLLING_INTERVAL=
# Interval between periodic flushes of open SiLK Flow files to disk,
# in seconds. This may be left blank, and will default to 120.
FLUSH_TIMEOUT=
# Maximum number of SiLK Flow files to have open for writing
# simultaneously. This may be left blank, and will default to 64
FILE_CACHE_SIZE=
# Whether rwflowpack should use advisory write locks. 1=yes, 0=no.
# Set to zero if messages like "Cannot get a write lock on file"
# appear in rwflowpack's log file.
FILE_LOCKING=1
# Whether rwflowpack should include the input and output SNMP
# interfaces and the next-hop-ip in the output files. 1=yes, 0=no.
# The default is no, and these values are not stored to save disk
# space. (The input and output fields contain VLAN tags when the
# sensor.conf file contains the attribute "interface-values vlan".)
PACK_INTERFACES=0
###
# The type of logging to use. Valid values are "legacy" and "syslog".
LOG_TYPE=syslog
# The lowest level of logging to actually log. Valid values are:
# emerg, alert, crit, err, warning, notice, info, debug
LOG_LEVEL=info
# The full path of the directory where the log files will be written
# when LOG_TYPE is "legacy".
LOG_DIR=/var/log
# The full path of the directory where the PID file will be written
PID_DIR=${LOG_DIR}
# The user this program runs as; root permission is required only when
# rwflowpack listens on a privileged port.
USER=root
#USER=`whoami` # run as user invoking the script
# Extra options to pass to rwflowpack
EXTRA_OPTIONS=
/etc/init.d/rwflowback directory, the only change was to line 38 in order to change to the configuration specified in the configure statement.
SCRIPT_CONFIG_LOCATION="/etc/silk"
With everything installed in their respective locations it is time to move on to setting up the Cisco device.
Router(config)# ip cef
Router(config)# ip flow-export source Loopback0
Router(config)# ip flow-export version 5
Router(config)# ip flow-export destination x.x.x.x 9990
Router(config)# interface f1/0
Router(config-if)# ip flow ingress
Router(config-if)# ip flow egress
I hope this helps. If you have any comments or questions, leave a comment below.
Comments
comments powered by Disqus