RHCE – Routing network traffic and configuring static routes

A CentOS machine can be set set up to act as router, i.e. it can be used facilitate the communication between two or more devices.

To understand how this works, we’ll use the following example:


       box1                       routingvm                        box2
+---------------------+      +----------------------+      +-------------------+
|                     |      |                      |      |                   |
|                     |      |                      |      |                   |
|  +---------------+  |      |    +---------------+ |      |                   |
|  |enp0s8:        |<-----------> |enp0s8:        | |      |                   |
|  |192.168.10.101 |  |      |    |192.168.10.100 | |      |                   |
|  |255.255.255.0  |  |      |    |255.255.255.0  | |      |                   |
|  +---------------+  |      |    +---------------+ |      |                   |
|                     |      |                      |      |                   |
|                     |      |                      |      |                   |
|                     |      |    +---------------+ |      |  +--------------+ |
|                     |      |    |enp0s9:        | |      |  | enp0s8:      | |
|                     |      |    |10.0.0.10      |<--------->| 10.0.0.11    | |
|                     |      |    |255.255.255.0  | |      |  | 255.255.255.0| |
|                     |      |    +---------------+ |      |  +--------------+ |
|                     |      |                      |      |                   |
|   +------------+    |      |    +-----------+     |      |  +----------+     |
|   |enp0s3      |    |      |    |enp0s3     |     |      |  | enp0s3   |     |
|   +-----+------+    |      |    +----+------+     |      |  +-----+----+     |
|         |           |      |         |            |      |        |          |
+---------------------+      +----------------------+      +-------------------+
          |                            |                            |
          v                            v                            v
+------------------------------------------------------------------------------+
|                        router (e.g. cisco router)                            |
|                                 (10.0.2.2)                                   |
+------------------------------------------------------------------------------+
                                       |
                                       v
+------------------------------------------------------------------------------+
|                                   INTERNET                                   |
|                                                                              |
+------------------------------------------------------------------------------+


The goal here is to get one centos box (box1) to successfully ping another centos box (box2). However the tricky part has to be directed to box2’s private ip address (10.0.0.11). This isn’t possible directly since box1 and box2 are on different private networks. That’s where the middle box (which I’ve named routingvm) comes into play. routingvm happens to be a member of both private networks, so it can potentially forward requests from box1 to box2.

Note: You can replicate the above setup on your local desktop using the cento7-static-routing-demo vagrant environment.

At this stage, when you ping box2 from box1, it will fail:

[root@box1 ~]# ping -c 3 10.0.0.11
PING 10.0.0.11 (10.0.0.11) 56(84) bytes of data.

--- 10.0.0.11 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2001ms

All three CentOS boxes can reach the internet, via their respective enp0s3 network adapters. the enp0s3 are designated for internet access only, so it won’t be possible to ping box2 via the internet, unless box2’s enp0s3 box has a dedicated public ip address attached to them. We’ll assume that isn’t the case, so let’s ignore all enp0s3 network adapters.

In the above setup there are 2 private internal networks, where:

– box1 (via the enp0s8 network) belongs to a private internal network, 192.168.10.0/24.
– box2 (via the enp0s8 network) belongs to a private internal network, 10.0.0.0/24.
– routingvm is a member of both of these private internal networks. It is a member of the 192.168.10.0/24, via the enp0s8 network adapter, and it is a member of 10.0.0.0/24 via the enp0s9 network adapter.

This means that routingvm can ping to both box1 and box2:

# pinging box1
[root@routingvm ~]# ping -c 3 192.168.10.101
PING 192.168.10.101 (192.168.10.101) 56(84) bytes of data.
64 bytes from 192.168.10.101: icmp_seq=1 ttl=64 time=0.268 ms
64 bytes from 192.168.10.101: icmp_seq=2 ttl=64 time=0.438 ms
64 bytes from 192.168.10.101: icmp_seq=3 ttl=64 time=0.306 ms

--- 192.168.10.101 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.268/0.337/0.438/0.074 ms

# pinging box2
[root@routingvm ~]# ping -c 3 10.0.0.11
PING 10.0.0.11 (10.0.0.11) 56(84) bytes of data.
64 bytes from 10.0.0.11: icmp_seq=1 ttl=64 time=0.280 ms
64 bytes from 10.0.0.11: icmp_seq=2 ttl=64 time=0.317 ms
64 bytes from 10.0.0.11: icmp_seq=3 ttl=64 time=0.300 ms

--- 10.0.0.11 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.280/0.299/0.317/0.015 ms

Unfortunately, the above output doesn’t show which interface the ping command used. But you can get that info using the ip command:

# routing info for box1
[root@routingvm ~]# ip route get 192.168.10.101
192.168.10.101 dev enp0s8  src 192.168.10.100
    cache

# routing info for box2
[root@routingvm ~]# ip route get 10.0.0.11
10.0.0.11 dev enp0s9  src 10.0.0.10
    cache

Also here’s the route table for routingvm:

[root@routingvm ~]# ip route list
default via 10.0.2.2 dev enp0s3  proto static  metric 100
10.0.0.0/24 dev enp0s9  proto kernel  scope link  src 10.0.0.10  metric 100
10.0.2.0/24 dev enp0s3  proto kernel  scope link  src 10.0.2.15  metric 100
192.168.10.0/24 dev enp0s8  proto kernel  scope link  src 192.168.10.100  metric 100

box1 can also ping routingvm:

[root@box1 ~]# ping -c 3 192.168.10.100
PING 192.168.10.100 (192.168.10.100) 56(84) bytes of data.
64 bytes from 192.168.10.100: icmp_seq=1 ttl=64 time=0.246 ms
64 bytes from 192.168.10.100: icmp_seq=2 ttl=64 time=0.272 ms
64 bytes from 192.168.10.100: icmp_seq=3 ttl=64 time=0.314 ms

--- 192.168.10.100 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.246/0.277/0.314/0.031 ms


[root@box1 ~]# ip route get 192.168.10.100
192.168.10.100 dev enp0s8  src 192.168.10.101
    cache


[root@box1 ~]# ip route list
default via 10.0.2.2 dev enp0s3  proto static  metric 100
10.0.2.0/24 dev enp0s3  proto kernel  scope link  src 10.0.2.15  metric 100
192.168.10.0/24 dev enp0s8  proto kernel  scope link  src 192.168.10.101  metric 100

Similarly box2 can also ping routingvm:

[root@box2 ~]# ping -c 3 10.0.0.10
PING 10.0.0.10 (10.0.0.10) 56(84) bytes of data.
64 bytes from 10.0.0.10: icmp_seq=1 ttl=64 time=0.375 ms
64 bytes from 10.0.0.10: icmp_seq=2 ttl=64 time=0.478 ms
64 bytes from 10.0.0.10: icmp_seq=3 ttl=64 time=0.325 ms

--- 10.0.0.10 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.325/0.392/0.478/0.067 ms


[root@box2 ~]# ip route get 10.0.0.10
10.0.0.10 dev enp0s8  src 10.0.0.11
    cache


[root@box2 ~]# ip route list
default via 10.0.2.2 dev enp0s3  proto static  metric 100
10.0.0.0/24 dev enp0s8  proto kernel  scope link  src 10.0.0.11  metric 100
10.0.2.0/24 dev enp0s3  proto kernel  scope link  src 10.0.2.15  metric 100

Now since routingvm can freely communicate with both box1 and box2, it means that routingvm can be used to facilitate network traffic between box1 and box2. To do this, you first need to configure the box1 network settings so that any traffic that is destined for the 10.0.0.0/24 network needs to sent via box1’s enp0s8 interface to routingvm.

This is done by running the following command to add a new rule to box1’s route table:

[root@box1 ~]# ip route add 10.0.0.0/24 via 192.168.10.100 dev enp0s8

Note: you can undo the above command by running it again, but with ‘add’ replaced with ‘del’

This ends up adding the following entry to the routing table:

[root@box1 ~]# ip route list
default via 10.0.2.2 dev enp0s3  proto static  metric 100
10.0.0.0/24 via 192.168.10.100 dev enp0s8
10.0.2.0/24 dev enp0s3  proto kernel  scope link  src 10.0.2.15  metric 100
192.168.10.0/24 dev enp0s8  proto kernel  scope link  src 192.168.10.101  metric 100

This line basically says, any traffic that is destined to an ip address that falls in the range 10.0.0.1-10.0.0.254, must be

However this won’t persistant, there are 3 ways to make it persistant, the first approach is to create this file

[root@box1 ~]# cat /etc/sysconfig/network-scripts/route-enp0s8
10.0.0.0/24 via 192.168.10.100

Note, you will need to create this file if it doesn’t already exist.

The second approach is to get the route-enp0s8 file created for you by using the nmcli command.

The third approach is to create the following file with the content:

$ cat /etc/sysconfig/static-routes
any net 10.0.0.0 netmask 255.255.255.0 gw 192.168.10.100 dev enp0s8

Note: /etc/sysconfig/static-routes is specifically mentioned in the network init script: /etc/init.d/network

After taking one of these 3 approaches, you need to restart the network daemon to load this in:

$ systemctl restart network

You can see the new rule by running:

[root@box1 ~]# ip route list
default via 10.0.2.2 dev enp0s3  proto static  metric 100
10.0.0.0/24 via 192.168.10.100 dev enp0s8
10.0.2.0/24 dev enp0s3  proto kernel  scope link  src 10.0.2.15  metric 100
192.168.10.0/24 dev enp0s8  proto kernel  scope link  src 192.168.10.101  metric 100

At this point we have now configured box1 to send box2-destined traffic to routingvm. But if we run:

[root@box1 ~]# traceroute 10.0.0.11
traceroute to 10.0.0.11 (10.0.0.11), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *

This still doesn’t look good. That’s because we need to configure routingvm to start acting as a router. To do this we need to first configure routingvm’s kernel setting to allow ip forwarding. First we check if it is currently enabled:

[root@routingvm ~]# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0

In this case, it isn’t, so we enable it:

[root@box1 ~]# /sbin/sysctl -w net.ipv4.ip_forward=1
net.ipv4.ip_forward = 1

The -w means we write this change to file in order to make it persistant. Now let’s confirm that has worked.

[root@routingvm ~]# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

Then we reload, to confirm that has worked:

[root@routingvm ~]# sysctl -p
[root@routingvm ~]# sysctl net.ipv4.ip_forward

Now if we run traceroute on box1 again, we get:

[root@box1 ~]# traceroute 10.0.0.11
traceroute to 10.0.0.11 (10.0.0.11), 30 hops max, 60 byte packets
 1  192.168.10.100 (192.168.10.100)  0.552 ms  0.506 ms  0.492 ms^C

This indicates that communication between box1 and routingvm is now working. However routingvm isn’t configured to forwarding on requests to box2 yet, so pinging is still failing:

[root@box1 ~]# ping -c3 10.0.0.11
PING 10.0.0.11 (10.0.0.11) 56(84) bytes of data.

--- 10.0.0.11 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2000ms

Next we start firewalld service. We need to use firewalld becuase it has an key feature for rerouting traffic, called ‘masquerade’:

[root@routingvm ~]# systemctl start firewalld
[root@routingvm ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: active (running) since Sat 2017-08-05 23:19:40 UTC; 2s ago
     Docs: man:firewalld(1)
 Main PID: 5925 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─5925 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid

Aug 05 23:19:40 routingvm.local systemd[1]: Starting firewalld - dynamic firewall daemon...
Aug 05 23:19:40 routingvm.local systemd[1]: Started firewalld - dynamic firewall daemon.

Finally we enabling the masquerade feature, reload the firewalld configs, and then confirm that it is enabled:

[root@routingvm ~]# firewall-cmd --permanent --add-masquerade
success
[root@routingvm ~]# firewall-cmd --reload
success
[root@routingvm ~]# firewall-cmd --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: enp0s3 enp0s8 enp0s9
  sources:
  services: dhcpv6-client ssh
  ports:
  protocols:
  masquerade: yes
  forward-ports:
  sourceports:
  icmp-blocks:
  rich rules:

Enabling masquerade basically means that firewall do will actively forward on any incoming network traffic that is destined for elsewhere.

That’s now everything that’s requires for configuring routingvm. We now go back to box1 and do another ping test:

[root@box1 ~]# ping -c 3 10.0.0.11
PING 10.0.0.11 (10.0.0.11) 56(84) bytes of data.
64 bytes from 10.0.0.11: icmp_seq=1 ttl=63 time=0.543 ms
64 bytes from 10.0.0.11: icmp_seq=2 ttl=63 time=0.821 ms
64 bytes from 10.0.0.11: icmp_seq=3 ttl=63 time=0.664 ms

--- 10.0.0.11 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.543/0.676/0.821/0.113 ms

Another test we can do is try to ssh into box2 from box1:

[root@box1 ~]# ssh 10.0.0.11
The authenticity of host '10.0.0.11 (10.0.0.11)' can't be established.
ECDSA key fingerprint is 25:f0:05:ec:12:ce:9f:33:ff:38:58:31:ad:e2:4d:bd.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.0.11' (ECDSA) to the list of known hosts.
root@10.0.0.11's password:
Last login: Sat Aug  5 15:56:05 2017
[root@box2 ~]#

Finally we can confirm that the traffic is going via routingvm by using the traceroute command:

[root@box1 ~]# traceroute 10.0.0.11
traceroute to 10.0.0.11 (10.0.0.11), 30 hops max, 60 byte packets
 1  192.168.10.100 (192.168.10.100)  0.306 ms  0.235 ms  0.146 ms
 2  10.0.0.11 (10.0.0.11)  0.853 ms  0.810 ms  0.728 ms

Recommend video courses

https://app.pluralsight.com/player?course=lfcs-linux-networking&author=andrew-mallett&name=lfcs-linux-networking-m6&clip=3&mode=live

For masquarading:

https://app.pluralsight.com/player?course=security-network-host-lfce&author=anthony-nocentino&name=security-network-host-lfce-m4&clip=8&mode=live