Virtual Private Network with FreeBSD (Part 2)

Standard

This is the second and the final part on setting up a private virtual network.  The part 1 has been uploaded about one week ago.

If you are following the guide in FreeBSD handbook, you must be frustrated because of the complicated configurations.  Those configurations are good in the sense it restricts the IPsec to particular IP pairs but they are also too tough for purpose of connecting just two routers.  In this example, I break these assumptions and go as minimal as possible so the readers can have a smoother learning curve.  I protect the private ports from unsolicited connections with the help of PF firewall instead of the IPsec configuration.

Step 7: Install Packages

Install “ipsec-tools” package on the routers.

pkg install ipsec-tools

Step 8: Configure IPsec Rules

Set the rules in IPsec in japao-roteador.  These two lines say:

  1. IP packets, in any protocols, from the Japan network (10.81.0.0) to the Singapore network (10.65.0.0) is sent out encapsulated in tunnel mode, from the japao-roteador (45.76.207.156) to cingapura-roteador (47.76.133.70), with the encryption mandatory.
  2. IP packets, in any protocols, from the Singapore network to the Japan network is accepted in encapsulated in tunnel mode, from cingapura-roteador to japao-roteador, with the encryption mandatory.
    # touch /usr/local/etc/racoon/setkey.conf
    # cat >> /usr/local/etc/racoon/setkey.conf << EOF
    spdadd 10.81.0.0/24 10.65.0.0/24 any -P out ipsec esp/tunnel/45.76.207.156-45.76.153.70/require;
    spdadd 10.65.0.0/24 10.81.0.0/24 any -P in ipsec esp/tunnel/45.76.153.70-45.76.207.156/require;
    EOF

On another router cingapura-roteador, the in and out are reversed for the obvious reason:

  1. Incoming: the Japan network to the Singapore network
  2. Outgoing: the Singapore network to the Japan network
  3. # touch /usr/local/etc/racoon/setkey.conf
    # cat >> /usr/local/etc/racoon/setkey.conf << EOF
    spdadd 10.81.0.0/24 10.65.0.0/24 any -P in ipsec esp/tunnel/45.76.207.156-45.76.153.70/require;
    spdadd 10.65.0.0/24 10.81.0.0/24 any -P out ipsec esp/tunnel/45.76.153.70-45.76.207.156/require;
    EOF

Please note I used the word “require” instead of “use” as written by some other documentations.  It means the message must be in the expected form to be accepted.  If the encryption is turned off accidentally, it should be disregarded rather than passing through.

Step 9: Configure Racoon

Racoon, as installed with the “ipsec-tools” package, is going to help us the IPsec negotiation phase.  Setting up the pre-shared keys is as easy as updating a file, with each line of the IP address, and then a secret string.  The file will be referenced by the Racoon script.  Setting up the Racoon main configuration can be much more straight forward than the handbook.  I copied an example in “/usr/local/share/ipsec-tools” and customised it.  The two routers will have very similar configuration except the listening IP.

  1. On japao-roteador:
    # touch /usr/local/etc/racoon/psk.txt
    # chmod 0600 /usr/local/etc/racoon/psk.txt
    # cat >> /usr/local/etc/racoon/psk.txt << EOF
    45.76.207.156 pleasedonotusethis
    EOF
    
    # cat > /usr/local/etc/racoon/racoon.conf << EOF
    path pre_shared_key "/usr/local/etc/racoon/psk.txt";
    log debug;
    
    listen
    {
      isakmp          45.76.153.70;
    }
    
    remote anonymous
    {
      exchange_mode     main,aggressive;
      lifetime          time 8 hour;
      passive           off;
      proposal_check    obey;
      nat_traversal     off;
      generate_policy   off;
      proposal {
        encryption_algorithm   3des;
        hash_algorithm         sha1;
        authentication_method  pre_shared_key;
        dh_group               1;
      }
    }
    sainfo anonymous
    {
      pfs_group                 1;
      lifetime                  time 12 hour;
      encryption_algorithm      3des;
      authentication_algorithm  hmac_sha1;
      compression_algorithm     deflate;
    }
    EOF
  2. On cingapura-roteador:
    # touch /usr/local/etc/racoon/psk.txt
    # chmod 0600 /usr/local/etc/racoon/psk.txt
    # cat > /usr/local/etc/racoon/psk.txt << EOF
    45.76.153.70 pleasedonotusethis
    EOF
    
    # cat > /usr/local/etc/racoon/racoon.conf << EOF
    path pre_shared_key "/usr/local/etc/racoon/psk.txt";
    log debug;
    
    listen
    {
      isakmp          45.76.207.156;
    }
    
    remote anonymous
    {
      exchange_mode     main,aggressive;
      lifetime          time 8 hour;
      passive           off;
      proposal_check    obey;
      nat_traversal     off;
      generate_policy   off;
      proposal {
        encryption_algorithm   3des;
        hash_algorithm         sha1;
        authentication_method  pre_shared_key;
        dh_group               1;
      }
    }
    sainfo anonymous
    {
      pfs_group                 1;
      lifetime                  time 12 hour;
      encryption_algorithm      3des;
      authentication_algorithm  hmac_sha1;
      compression_algorithm     deflate;
    }
    EOF

Step 10: Enable Service

Almost finish.  We enable the services and start them as soon as possible.

# cat >> /etc/rc.conf << EOF
racoon_enable="YES"
ipsec_enable="YES"
ipsec_program="/usr/local/sbin/setkey"
ipsec_file="/usr/local/etc/racoon/setkey.conf"
EOF
# service racoon start
# service ipsec start

Step 11: Update Firewall Rules

We have been using firewall.  In the step 3 of part 1, we allowed peers to pass through with encapsulated messages.  Now, we let a few more items to go through: the authentication AH packets, the encapsulated ESP messages.

# cat >> /etc/rc.conf << EOF
pass in quick inet proto {ah,esp,ipencap} from $encappeers to any
pass in quick inet proto udp from $encappeers port = 500 to any port = 500
EOF

Step 12: Try to Break Something

Hopefully, things should go well when you ping the two hosts.  If not, you can watch the debug log “/var/log/debug.log” for a bit more insights.  The next step is to break the tunnel.  Here are some examples.

  1. Try stopping Racoon on one of the routers
  2. Try stopping IPsec on one of the routers
  3. Try using different pre-shared keys on the two routers
  4. Repeat step 1, 2, 3 with “use” rather than “require” in the IPsec configuration
  5. Deleting the route on one of the routers

Virtual Private Network with FreeBSD (Part 1)

Standard

With inexpensive and agile cloud services, it is not unusual for a service to have multiple points of presence around the globe.  In each point of presence, there is usually a per-tenant private network for internal (and confidential) system communications.  One can go further and combine these internal networks together with virtual links.   Here I explain the steps how this can be achieved with FreeBSD without relying extra networking services from the cloud vendors.

The article is split into two parts due to the length.  The first part, this, is about creating a tunnel with ip-in-ip encapsulation.  The second part is about encrypting the communications between the two hosts.

Concepts

Let’s say we have two private networks, network 1 (10.81.x.x) in Japan, and network 2 (10.65.x.x) in Singapore.  The two networks are connected through routers in the public internet.  However, these routers are not acknowledged about our own private network addresses, so we cannot send our messages directly between the two sites.  To enable private communication between the two sites, we will need network tunnel technique and network encryption.

A tunnel can be created by establishing one host on each private network as a router.  Each of these routers have a public routable network address.  Whenever communications between the two private networks have to be done, the network packets are sent through their respective routers.  The routers encapsulate the packets in new packets with “proper” addresses, so that the public routers understand how to route.  Once the encapsulated packets arrive the destination router, it is decapsulated for internal forwarding.  We deploy the FreeBSD generic tunnel interfaces for this part.

The messages between the two private networks are usually secret and thus need to be encrypted.  We can configure the routers so that whenever they communicate each other, the network packets have to be encrypted.  In the second half of this topic, we will employ IPSec (alternative link) for this purpose.

Notations

In this article, the commands are prepared in preformatted text, with pound (#) sign in front of each.  There are two types of commands, one that configures the system settings permanently and one that configures the current prevalent configurations.  The former are usually in sysrc(8) and cat(1) commands.  The latter are usually ifconfig(8) and sysctl(8).

Step 0: Environment

In this setup, we will prepare two private networks, each with a end client and a router.  I use the most inexpensive plans as possible.  As of today (Feb 2017), it is common to find virtual machines around 5 US dollars per month.  I used Vultr for the demonstration and they provide virtual machines at 768 MB for 5 dollars a month.  I assume you apply my customisation script for a minimal system and firewall configuration.

  1. Host 1: japao-roteador
    • Site: Tokyo
    • Plan: anything with 512 MB memory
    • Options: enable external and internal network
    • External interface: vtnet0 (DHCP assigned: 45.76.207.156)
    • Internal interface: vtnet1 (inet 10.81.0.1  netmask 255.255.255.0)
      # sysrc ifconfig_vtnet1="inet 10.81.0.1 netmask 255.255.255.0"
      # ifconfig vtnet1 inet 10.81.0.1 netmask 255.255.255.0
  2. Host 2: japao-host
    • Site: Tokyo
    • Plan: anything with 512 MB memory
    • Options: enable internal network
    • External interface: vtnet0 (DHCP assigned: 45.32.11.250)
    • Internal interface: vtnet1 (inet 10.81.0.11  netmask 255.255.255.0)
      # sysrc ifconfig_vtnet1="inet 10.81.0.11 netmask 255.255.255.0"
      # ifconfig vtnet1 inet 10.81.0.11 netmask 255.255.255.0
  3. Host 3: cingapura-roteador
    • Site: Singapore
    • Plan: anything with 512 MB memory
    • Options: enable external and internal network
    • External interface: vtnet0 (DHCP assigned: 45.76.153.70)
    • Internal interface: vtnet1 (inet 10.65.0.1  netmask 255.255.255.0)
      # sysrc ifconfig_vtnet1="inet 10.65.0.1 netmask 255.255.255.0"
      # ifconfig vtnet1 inet 10.65.0.1 netmask 255.255.255.0
  4. Host 4: cingapura-host
    • Site: Singapore
    • Plan: anything with 512 MB memory
    • Options: enable internal network
    • External interface: vtnet0 (DHCP assigned: 45.76.145.49)
    • Internal interface vtnet1 (inet 10.65.0.11  netmask 255.255.255.0)
      # sysrc ifconfig_vtnet1="inet 10.65.0.11 netmask 255.255.255.0"
      # ifconfig vtnet1 inet 10.65.0.11 netmask 255.255.255.0

Step 1: Tunnel and Route between the Routers

Next step, we will need to turn the designated hosts as routers, create generic tunnel network interface (gif) each of the routers, and set the route accordingly.

  1. Host 1: japao-roteador
    1. Internal: 10.81.0.1 (local) to 10.65.0.1 (counterpart)
    2. External: 45.76.207.156 (local) to 45.76.153.70 (counterpart)
    3. Route: 10.65.0.0/24 (network in Singapore) to 10.65.0.1 (router in Singapore)
      # sysrc gateway_enable="YES"
      # sysrc cloned_interfaces+="gif0"
      # sysrc ifconfig_gif0="10.81.0.1 10.65.0.1 netmask 255.255.255.0"
      # sysrc ifconfig_gif0+="tunnel 45.76.207.156 45.76.153.70"
      # sysrc static_routes+="cingapura"
      # sysrc route_cingapura="10.65.0.0/24 10.65.0.1"
      
      
      # sysctl net.inet.ip.forwarding=1
      # ifconfig gif0 create
      # ifconfig gif0 10.81.0.1 10.65.0.1 netmask 255.255.255.0
      # ifconfig gif0 tunnel 45.76.207.156 45.76.153.70
      # route add 10.65.0.1/24 10.65.0.1
  2. Host 3: cingapura-roteador
    1. Internal: 10.65.0.1 (local) to 10.81.0.1 (counterpart)
    2. External: 45.76.153.70 (local) to 45.76.207.156 (counterpart)
    3. Route: 10.81.0.0/24 (network in Japan) to 10.81.0.1 (router in Japan)
      # sysrc gateway_enable="YES"
      # sysrc cloned_interfaces+="gif0"
      # sysrc ifconfig_gif0="10.65.0.1 10.81.0.1 netmask 255.255.255.0"
      # sysrc ifconfig_gif0+="tunnel 45.76.153.70 45.76.207.156"
      # sysrc static_routes+="japao"
      # sysrc route_japao="10.81.0.0/24 10.81.0.1"
      
      # sysctl net.inet.ip.forwarding=1
      # ifconfig gif0 create
      # ifconfig gif0 10.65.0.1 10.81.0.1 netmask 255.255.255.0
      # ifconfig gif0 tunnel 45.76.153.70 45.76.207.156
      # route add 10.81.0.1/24 10.81.0.1

Step 2: Update Firewall Rules of the Routers

You will notice that after setting the items above, the router in Singapore still cannot obtain ping reply from the internal network address of the router in Japan (10.81.0.1), and vice versa.  If the pf firewall on one side is disabled, it works.  This is because the computers are having a firewall enabled (per our standard customisation script).  Let us relax the rules to let these through.

  1. Host 1: japao-roteador
    1. Counterpart peer: cingapura-roteador (45.76.153.70)
    2. Protocol to allow: ipencap
      # cat >> /etc/pf.conf << EOF
      encappeers="{45.76.153.70}"
      pass in quick inet proto ipencap from $encappeers to any
      EOF
      # service pf reload
  2. Host 2: cingapura-roteador
    1. Counterpart peer: japao-roteador (45.76.207.156)
    2. Protocol to allow: ipencap
      # cat >> /etc/pf.conf << EOF
      encappeers="{45.76.207.156}"
      pass in quick inet proto ipencap from $encappeers to any
      EOF
      # service pf reload

 

Step 3: Configure Routing on the Clients

Similarly, we add static routes to the client hosts.  Please note they are subtlety different.  For example:  The router in Japan is directing Singapore traffic to the router in Singapore.  The host in Japan is directing Singapore traffic to the router in Japan instead.

  1. Host 2: japao-host
    1. Route: 10.65.0.0/24 (network in Singapore) to 10.81.0.1 (router in Japan)
      # sysrc static_routes+="cingapura"
      # sysrc route_cingapura="10.65.0.0/24 10.81.0.1"
      # route add 10.65.0.0/24 10.81.0.1
  2. Host 4: cingapura-host
    1. Route: 10.81.0.0/24 (network in Japan) to 10.81.0.1 (router in Singapore)
      # sysrc static_routes+="japao"
      # sysrc route_cingapura="10.81.0.0/24 10.65.0.1"
      # route add 10.81.0.0/24 10.65.0.1

Step 4: Testing

  1. To test the routers, use command ping from japao-reteador to 10.65.0.1 (Singapore), and from cingapura-roteador to 10.81.0.1 (Japan).  Nice.  The two sites are 76 milliseconds apart.
    root@japao-roteador ~# ping 10.65.0.1
    PING 10.65.0.1 (10.65.0.1): 56 data bytes
    64 bytes from 10.65.0.1: icmp_seq=0 ttl=64 time=76.978 ms
    64 bytes from 10.65.0.1: icmp_seq=1 ttl=64 time=76.898 ms
    64 bytes from 10.65.0.1: icmp_seq=2 ttl=64 time=76.859 ms
    64 bytes from 10.65.0.1: icmp_seq=3 ttl=64 time=76.903 ms
    64 bytes from 10.65.0.1: icmp_seq=4 ttl=64 time=76.758 ms
    64 bytes from 10.65.0.1: icmp_seq=5 ttl=64 time=77.120 ms
    ^C
    --- 10.65.0.1 ping statistics ---
    6 packets transmitted, 6 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 76.758/76.919/77.120/0.111 ms
  2. To test the hosts, use command ping from japao-host to 10.65.0.11 (Singapore host), and from cingapura-host to 10.81.0.11 (Japan host).  There is slightly one more millisecond in the round trip time.
    root@japao-host ~# ping 10.65.0.11
    PING 10.65.0.11 (10.65.0.11): 56 data bytes
    64 bytes from 10.65.0.11: icmp_seq=0 ttl=62 time=77.892 ms
    64 bytes from 10.65.0.11: icmp_seq=1 ttl=62 time=77.817 ms
    64 bytes from 10.65.0.11: icmp_seq=2 ttl=62 time=78.710 ms
    64 bytes from 10.65.0.11: icmp_seq=3 ttl=62 time=78.227 ms
    64 bytes from 10.65.0.11: icmp_seq=4 ttl=62 time=78.444 ms
    ^C
    --- 10.65.0.11 ping statistics ---
    5 packets transmitted, 5 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 77.817/78.218/78.710/0.335 ms

Step 5: Troubleshooting

I hope you do not need this step.  Anyway…

When you are unable to receive a ping reply, there are mostly two reasons.  Either the ping request is unable to reach the destination, or the ping reply could not transverse back to the requestor.  Seeing the packets on both the hosts simultaneously gives you insight which case it is.  To see if there are any traffic in the tunnel, issue on the routers:

# tcpdump -i gif0

Why is a packet unable to appear in the network?  There are mostly two reasons.  Either the network firewall blocks it (unlikely if you follow my instructions) or the route is set incorrectly.  To see if the firewall state table and the route table:

# pfctl -s states
# netstat -rn

Step 6: To be Continued

At this stage, when you use “tcpdump” on the interface, you get messages like:

# tcpdump host 45.76.153.70
16:09:26.759030 IP 45.76.207.156.vultr.com > 45.76.153.70.vultr.com: IP 10.81.0.1 > 10.65.0.1: ICMP echo reply, id 53509, seq 9, length 64 (ipip-proto-4)

And we want it to be encrypted, like:

# tcpdump host 45.76.153.70
16:09:32.314383 IP 45.76.207.156.vultr.com > 45.76.153.70.vultr.com: ESP(spi=0x004a1fe7,seq=0x1c), length 116

In the next part, I will go through the steps to encrypt the tunnel.