This post demonstrates how you can mirror interfaces on a Linux server in an environment where you may not have physical network taps or SPAN ports. We can use OpenvSwitch in order to forward traffic between nodes, even if we are not using virtualization. Each node being monitored needs two interfaces, one for production traffic and the other being an internal or mirrored interface where you send traffic to be aggregated and analyzed by your cloud based security stack. You will need to be cognizant of the amount of data you are sending to your aggregation point as it may become saturated if you send traffic from multiple nodes that exceeds the receiving nodes capacity.

On VM to have a monitored interface:

Ensure the host has two network interfaces and determine which one is production verse management. The management interface will be used to send traffic to your aggregation or collection node as previously described above. For this example, eth0 and eth1 are production and management respectively.

Install OpenvSwitch:

$ sudo apt-get install openvswitch-switch

Bring up the secondary interface, we will use this as the bridge interface, i.e. the interface that sends mirrored eth0 traffic:

$ sudo ifconfig eth1 172.31.3.110 netmask 255.255.240.0

Configure bridge and set remote IP to your collection node which is a different network (interface) then that which is being mirrored:

$ sudo ovs-vsctl add-br br0
$ sudo ovs-vsctl add-port br0 eth1
$ sudo ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre options:remote_ip=172.31.10.151 -- --id=@p get port gre0 -- --id=@m create mirror name=m0 select-all=true output-port=@p -- set bridge br0 mirrors=@m

The following steps will disconnect you from eth0 so may be ideal to connect to eth1 at this point or respectively your bridge interface. Null the network address to be mirrored and set the IP to that of the bridge interface as well as updating the gateway. We also assign the bridge interface to the MAC address of eth0 as some environments may not allow traffic to/from interfaces hardware addresses they do not know about.

$ sudo ifconfig br0 172.31.11.64 netmask 255.255.240.0 
$ sudo ifconfig eth0 0
$ sudo ifconfig br0 hw ether 0a:74:0c:89:fb:70
$ sudo route add default gw 172.31.0.1 br0

We can now view the mirrored traffic on the host defined at the remote IP, packets are encapsulated but you may see protocol unreachable ICMP messages. This is because br0 drops responses. The next step fixes this by completing/terminating the tunnel on the remote host which will unencapsulate the GRE tunnel. Here, we again use eth0 and eth1 as production and management networks but we do not have to. We could just have one interface that accepts traffic from the clients forwarding us their network traffic but if it becomes saturated it may be difficult to connect to the host.

$ sudo ifconfig eth1 172.20.1.7 netmask 255.255.255.240
$ sudo modprobe ip_gre
$ sudo lsmod | grep ip_gre
$ sudo ip tunnel add mon0 mode gre local 172.20.1.7 remote 
$ sudo ip addr add 1.1.1.1/30 dev mon0
$ sudo ip link set mon0 up

Now you can monitor interface mon0 using tools like tcpdump or simply capture network traffic for retroactive analysis.

If you need to, remove the bridge and port using the following commands:

$ sudo ovs-vsctl clear bridge br0 mirrors
$ sudo ovs-vsctl del-port br0 gre0