Virtual Private Network with FreeBSD (Part 1)

Standard

With inexpensive and agile cloud services, it is not unusual for a service to have multiple points of presence around the globe.  In each point of presence, there is usually a per-tenant private network for internal (and confidential) system communications.  One can go further and combine these internal networks together with virtual links.   Here I explain the steps how this can be achieved with FreeBSD without relying extra networking services from the cloud vendors.

The article is split into two parts due to the length.  The first part, this, is about creating a tunnel with ip-in-ip encapsulation.  The second part is about encrypting the communications between the two hosts.

Concepts

Let’s say we have two private networks, network 1 (10.81.x.x) in Japan, and network 2 (10.65.x.x) in Singapore.  The two networks are connected through routers in the public internet.  However, these routers are not acknowledged about our own private network addresses, so we cannot send our messages directly between the two sites.  To enable private communication between the two sites, we will need network tunnel technique and network encryption.

A tunnel can be created by establishing one host on each private network as a router.  Each of these routers have a public routable network address.  Whenever communications between the two private networks have to be done, the network packets are sent through their respective routers.  The routers encapsulate the packets in new packets with “proper” addresses, so that the public routers understand how to route.  Once the encapsulated packets arrive the destination router, it is decapsulated for internal forwarding.  We deploy the FreeBSD generic tunnel interfaces for this part.

The messages between the two private networks are usually secret and thus need to be encrypted.  We can configure the routers so that whenever they communicate each other, the network packets have to be encrypted.  In the second half of this topic, we will employ IPSec (alternative link) for this purpose.

Notations

In this article, the commands are prepared in preformatted text, with pound (#) sign in front of each.  There are two types of commands, one that configures the system settings permanently and one that configures the current prevalent configurations.  The former are usually in sysrc(8) and cat(1) commands.  The latter are usually ifconfig(8) and sysctl(8).

Step 0: Environment

In this setup, we will prepare two private networks, each with a end client and a router.  I use the most inexpensive plans as possible.  As of today (Feb 2017), it is common to find virtual machines around 5 US dollars per month.  I used Vultr for the demonstration and they provide virtual machines at 768 MB for 5 dollars a month.  I assume you apply my customisation script for a minimal system and firewall configuration.

  1. Host 1: japao-roteador
    • Site: Tokyo
    • Plan: anything with 512 MB memory
    • Options: enable external and internal network
    • External interface: vtnet0 (DHCP assigned: 45.76.207.156)
    • Internal interface: vtnet1 (inet 10.81.0.1  netmask 255.255.255.0)
      # sysrc ifconfig_vtnet1="inet 10.81.0.1 netmask 255.255.255.0"
      # ifconfig vtnet1 inet 10.81.0.1 netmask 255.255.255.0
  2. Host 2: japao-host
    • Site: Tokyo
    • Plan: anything with 512 MB memory
    • Options: enable internal network
    • External interface: vtnet0 (DHCP assigned: 45.32.11.250)
    • Internal interface: vtnet1 (inet 10.81.0.11  netmask 255.255.255.0)
      # sysrc ifconfig_vtnet1="inet 10.81.0.11 netmask 255.255.255.0"
      # ifconfig vtnet1 inet 10.81.0.11 netmask 255.255.255.0
  3. Host 3: cingapura-roteador
    • Site: Singapore
    • Plan: anything with 512 MB memory
    • Options: enable external and internal network
    • External interface: vtnet0 (DHCP assigned: 45.76.153.70)
    • Internal interface: vtnet1 (inet 10.65.0.1  netmask 255.255.255.0)
      # sysrc ifconfig_vtnet1="inet 10.65.0.1 netmask 255.255.255.0"
      # ifconfig vtnet1 inet 10.65.0.1 netmask 255.255.255.0
  4. Host 4: cingapura-host
    • Site: Singapore
    • Plan: anything with 512 MB memory
    • Options: enable internal network
    • External interface: vtnet0 (DHCP assigned: 45.76.145.49)
    • Internal interface vtnet1 (inet 10.65.0.11  netmask 255.255.255.0)
      # sysrc ifconfig_vtnet1="inet 10.65.0.11 netmask 255.255.255.0"
      # ifconfig vtnet1 inet 10.65.0.11 netmask 255.255.255.0

Step 1: Tunnel and Route between the Routers

Next step, we will need to turn the designated hosts as routers, create generic tunnel network interface (gif) each of the routers, and set the route accordingly.

  1. Host 1: japao-roteador
    1. Internal: 10.81.0.1 (local) to 10.65.0.1 (counterpart)
    2. External: 45.76.207.156 (local) to 45.76.153.70 (counterpart)
    3. Route: 10.65.0.0/24 (network in Singapore) to 10.65.0.1 (router in Singapore)
      # sysrc gateway_enable="YES"
      # sysrc cloned_interfaces+="gif0"
      # sysrc ifconfig_gif0="10.81.0.1 10.65.0.1 netmask 255.255.255.0"
      # sysrc ifconfig_gif0+="tunnel 45.76.207.156 45.76.153.70"
      # sysrc static_routes+="cingapura"
      # sysrc route_cingapura="10.65.0.0/24 10.65.0.1"
      
      
      # sysctl net.inet.ip.forwarding=1
      # ifconfig gif0 create
      # ifconfig gif0 10.81.0.1 10.65.0.1 netmask 255.255.255.0
      # ifconfig gif0 tunnel 45.76.207.156 45.76.153.70
      # route add 10.65.0.1/24 10.65.0.1
  2. Host 3: cingapura-roteador
    1. Internal: 10.65.0.1 (local) to 10.81.0.1 (counterpart)
    2. External: 45.76.153.70 (local) to 45.76.207.156 (counterpart)
    3. Route: 10.81.0.0/24 (network in Japan) to 10.81.0.1 (router in Japan)
      # sysrc gateway_enable="YES"
      # sysrc cloned_interfaces+="gif0"
      # sysrc ifconfig_gif0="10.65.0.1 10.81.0.1 netmask 255.255.255.0"
      # sysrc ifconfig_gif0+="tunnel 45.76.153.70 45.76.207.156"
      # sysrc static_routes+="japao"
      # sysrc route_japao="10.81.0.0/24 10.81.0.1"
      
      # sysctl net.inet.ip.forwarding=1
      # ifconfig gif0 create
      # ifconfig gif0 10.65.0.1 10.81.0.1 netmask 255.255.255.0
      # ifconfig gif0 tunnel 45.76.153.70 45.76.207.156
      # route add 10.81.0.1/24 10.81.0.1

Step 2: Update Firewall Rules of the Routers

You will notice that after setting the items above, the router in Singapore still cannot obtain ping reply from the internal network address of the router in Japan (10.81.0.1), and vice versa.  If the pf firewall on one side is disabled, it works.  This is because the computers are having a firewall enabled (per our standard customisation script).  Let us relax the rules to let these through.

  1. Host 1: japao-roteador
    1. Counterpart peer: cingapura-roteador (45.76.153.70)
    2. Protocol to allow: ipencap
      # cat >> /etc/pf.conf << EOF
      encappeers="{45.76.153.70}"
      pass in quick inet proto ipencap from $encappeers to any
      EOF
      # service pf reload
  2. Host 2: cingapura-roteador
    1. Counterpart peer: japao-roteador (45.76.207.156)
    2. Protocol to allow: ipencap
      # cat >> /etc/pf.conf << EOF
      encappeers="{45.76.207.156}"
      pass in quick inet proto ipencap from $encappeers to any
      EOF
      # service pf reload

 

Step 3: Configure Routing on the Clients

Similarly, we add static routes to the client hosts.  Please note they are subtlety different.  For example:  The router in Japan is directing Singapore traffic to the router in Singapore.  The host in Japan is directing Singapore traffic to the router in Japan instead.

  1. Host 2: japao-host
    1. Route: 10.65.0.0/24 (network in Singapore) to 10.81.0.1 (router in Japan)
      # sysrc static_routes+="cingapura"
      # sysrc route_cingapura="10.65.0.0/24 10.81.0.1"
      # route add 10.65.0.0/24 10.81.0.1
  2. Host 4: cingapura-host
    1. Route: 10.81.0.0/24 (network in Japan) to 10.81.0.1 (router in Singapore)
      # sysrc static_routes+="japao"
      # sysrc route_cingapura="10.81.0.0/24 10.65.0.1"
      # route add 10.81.0.0/24 10.65.0.1

Step 4: Testing

  1. To test the routers, use command ping from japao-reteador to 10.65.0.1 (Singapore), and from cingapura-roteador to 10.81.0.1 (Japan).  Nice.  The two sites are 76 milliseconds apart.
    root@japao-roteador ~# ping 10.65.0.1
    PING 10.65.0.1 (10.65.0.1): 56 data bytes
    64 bytes from 10.65.0.1: icmp_seq=0 ttl=64 time=76.978 ms
    64 bytes from 10.65.0.1: icmp_seq=1 ttl=64 time=76.898 ms
    64 bytes from 10.65.0.1: icmp_seq=2 ttl=64 time=76.859 ms
    64 bytes from 10.65.0.1: icmp_seq=3 ttl=64 time=76.903 ms
    64 bytes from 10.65.0.1: icmp_seq=4 ttl=64 time=76.758 ms
    64 bytes from 10.65.0.1: icmp_seq=5 ttl=64 time=77.120 ms
    ^C
    --- 10.65.0.1 ping statistics ---
    6 packets transmitted, 6 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 76.758/76.919/77.120/0.111 ms
  2. To test the hosts, use command ping from japao-host to 10.65.0.11 (Singapore host), and from cingapura-host to 10.81.0.11 (Japan host).  There is slightly one more millisecond in the round trip time.
    root@japao-host ~# ping 10.65.0.11
    PING 10.65.0.11 (10.65.0.11): 56 data bytes
    64 bytes from 10.65.0.11: icmp_seq=0 ttl=62 time=77.892 ms
    64 bytes from 10.65.0.11: icmp_seq=1 ttl=62 time=77.817 ms
    64 bytes from 10.65.0.11: icmp_seq=2 ttl=62 time=78.710 ms
    64 bytes from 10.65.0.11: icmp_seq=3 ttl=62 time=78.227 ms
    64 bytes from 10.65.0.11: icmp_seq=4 ttl=62 time=78.444 ms
    ^C
    --- 10.65.0.11 ping statistics ---
    5 packets transmitted, 5 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 77.817/78.218/78.710/0.335 ms

Step 5: Troubleshooting

I hope you do not need this step.  Anyway…

When you are unable to receive a ping reply, there are mostly two reasons.  Either the ping request is unable to reach the destination, or the ping reply could not transverse back to the requestor.  Seeing the packets on both the hosts simultaneously gives you insight which case it is.  To see if there are any traffic in the tunnel, issue on the routers:

# tcpdump -i gif0

Why is a packet unable to appear in the network?  There are mostly two reasons.  Either the network firewall blocks it (unlikely if you follow my instructions) or the route is set incorrectly.  To see if the firewall state table and the route table:

# pfctl -s states
# netstat -rn

Step 6: To be Continued

At this stage, when you use “tcpdump” on the interface, you get messages like:

# tcpdump host 45.76.153.70
16:09:26.759030 IP 45.76.207.156.vultr.com > 45.76.153.70.vultr.com: IP 10.81.0.1 > 10.65.0.1: ICMP echo reply, id 53509, seq 9, length 64 (ipip-proto-4)

And we want it to be encrypted, like:

# tcpdump host 45.76.153.70
16:09:32.314383 IP 45.76.207.156.vultr.com > 45.76.153.70.vultr.com: ESP(spi=0x004a1fe7,seq=0x1c), length 116

In the next part, I will go through the steps to encrypt the tunnel.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s