Advanced Site to Site VPN.

Objectives:

1. Create and test a VPN connection between an AWS VPC and an On Premise Customer network.
2. Configure IPSec tunnels and understand BGP route propagation.

Architecture: The AWS VPC has 2 subnets in 2 AZs with an EC1 instance in each subnet. On the Customer side there are 2 physical routers and 2 servers. Connection is implemented using a Transit Gateway on the AWS side with 2 VPN attachments. This provides us 2 endpoints on AWS side which will connect to the two Customer Routers. This provides resiliency in case of failure of a customer router or failure of an AZ on the AWS side.

For practicality the customer network is simulated and is hosted on AWS. On the customer side the IPsec tunnels will be setup and configured using StrongSwan. StrongSwan is mature and widely adopted open-sourced IPsec based solution that provides secure communication and VPN capabilities. It supports various authentication methods, such as X.509 certificates, pre-shared keys, and username/password authentication. It adheres to industry standard protocols and specifications including IPsec RFCs, IKE RFCs and other relevant standards. Dynamic routing with BGP is set up using Free Range Routing (FRR), an open source routing software suite. Here FRR is used in conjunction with Strongswan to enhance the routing capabilities of this VPN solution.

A Site to Site VPN has the following components: AWS side: 1. A VPC 2. A Virtual Private Gateway on the AWS side - this is a logical gateway which would be the target of one or more route tables. Customer Side: 1. A Customer Network (On Prem) 2. A Customer Gateway on the Customer side, which can be the logical component within AWS and the physical router on the Customer side that this logical piece connects to. . Common to both sides: The tunnel through which data flows.

Steps :

1. Deploy the AWS VPC and a simulated On Prem Customer network using one click CF deployment. After setup, there is no connection between the AWS VPC and the On Prem environment which we will verify prior to setting up the VPN connection. Details
2. Create the Customer Gateway which is the logical connection that represent the physical Customer Router. These connections tell AWS how to connect to the routers using public IP addressing. Since there are 2 Physical Routers on the Customer side, there needs to be 2 Customer Gateways. Details
3. Verify there is no connection between the VPC and Customer network. Details
4. Create the VPN attachments which provide 2 endpoints. There is already 1 Transit Gateway attachment that connects the TGW to the AWS VPC. For the 2 new ones we will select the accelerated VPN option. This creates 2 accelerated endpoints per connection. These endpoints provide transit back to the AWS network via the AWS global network. The effect of this is that we will have two Site to Site VPN connections which can be used to connect the VPN endpoints to the On Premises router using IPsec tunnels, 2 per connection. Details
5. Download the configuration parameters for the Site To Site VPN connections. Each of these correspond to one Customer Gateway. This is needed to set up the Strongswan VPN appliance. Extract IP address, shared secret and other relevant information from the configuration files. This is needed to configure the VPN tunnels and to configure BGP. Tunnels are created using the same pre shared key. This key is one of the configuration parameters in the downloaded file. Inside the tunnels - here the internal IP addresses are used. This is where BGP traffic runs and this is the tunnel through which data is transmitted. To emphasise: Outside tunnel refers to the encrypted data stream between both parties, the IKE phase 1 tunnel. Inside Tunnel refers to the routing and the raw data that is transmitted, the IKE Phase 2 tunnel. Each Customer Gateway connects to two endpoints on the AWS side for high availability which has one tunnel each. Extract values from the configuration file. The inside IP, outside IP, shared secret etc. Details
6. Configure Strongswan using these parameters. Make sure both VPN connections are available. The resources (ONPREM-ROUTER1 and ONPREM-ROUTER2) that were deployed using the one click deployment script already came with scripts used to configure Strongswan. In the real world we would have to install a VPN solution ourselves. Modify the files and copy them to the /etc directory where they will be read by the strongswan software. Restart strongswan. After the setup we should be able to see two virtual tunnel interfaces. This means that the tunnel interfaces are active and are connected to AWS. Do the same for ONPREM-ROUTER2. After this is complete we should see that IPSEC is UP for both tunnels in Both Site-to-Site VPNS. Details
7. Configure BGP to run on top of the IPsec tunnels and establish connectivity between the two parties. This is done using FRR, Free Range Routing which provides a solution for routing protocol implementations, in our case BGP. This is how we are configuring dynamic routing between the networks. FRR exchanges routing information with other routers in the network, enabling efficient and dynamic route propagation. Details
8. Once the BGP configuration is done we can ping each other from both sides. Details
9. Clean up by deleting the resources in order and finally deleting the CF stack. Details
10. Summary & What I learned. Details

Detailed implementation steps:

Step1:

When the one click deployment is complete we have the VPC created, with CIDR 10.16.0.0/16 The On Prem CIDR is 192.168.8.0/21

Step 2:

Create Customer Gateway Objects

Step 3:

Verify connectivity. There is no connectivity.

Step 4:

Create VPN attachments:

Step 5:

Download connection parameters from the VPN Site To Site Connection for the appropriate attachment.

• The AWS side is now complete. We've configured the endpoints that the Customer can connect to. We've downloaded the configuration information for those endpoints.
• The On Prem setup has 2 layers - the IPsec and the BGP layer. We are now going to configure StrongSwan for IPsec.
• The On Prem router that came with the CF template in this exercise came preinstalled with Strongswan software. In the real world we will need to install the IPN solution ourselves and configure it based on the downloaded file. Make sure to pick the correct configuration based on the chosen vendor.

Step 6:

Configure Strongswan
ipsec.conf contains the actual configuration for the IPsec tunnels.
ipsec.secrets has the authentication information that this linux server will use to authenticate against aws.
ipsec-vti.sh script file - this will enable and disable the IPsec tunnel whenever the system detects traffic.

• These files need to be edited using parameters from the downloaded config files.
• This needs to be done carefully where the AWS inside ip address, AWS outside IP address, shared secret for the phase 1 IPsec tunnel is substituted correctly.
• After this is complete and the script is run, IKE Phase 1 tunnel gets setup and the ipsec-vti.sh script will enable the IKE phase 2 IPsec tunnel whenever the system detects traffic and tear it down when there is no interesting traffic.
  systemctl restart strongswan will bring both the IPsec tunnels from the Customer side to the AWS side. After everything has been configured and after Strongswan has been rebooted, two vti interfaces would show up on ifconfig.
  

Now if we check Site to Site VPN we should see that their IPSEC IS UP. This means these tunnels are active and that connectivity from On Prem routers to our VPC is working and we can use these tunnels to establish BGP sessions and route traffic through them.

Step 7:

In order to configure BGP to run on top of these tunnels, install FRR on both routers. SSH into Router 1. The router that came with the one click deployment came already setup with the script. Again in real world scenario, we would need to do install this ourselves.

Wait for the setup to complete.

Step 8:

Configure BGP.

Once the Router is back it will be functioning as both an IPsec endpoint and a BGP endpoint. It will now start exchanging routes with the Transit Gateway in the VPC. Now onshow ip route the VPC ip address is displayed.

Now to make sure pings are working from both sides, log on to VPC EC2 server and ping On Prem server.

Log on to On Prem server and ping VPC EC2:

Repeat Step 7 to install FRR on Router 2 and configure BGP. Verify that the ping is working.

Step 9:

Cleanup:
Delete VPN Connections

Delete Customer Gateways

Delete CloudFormation stack.

Summary

What did I learn?

1. Implemented a complex architecture with IPSec Tunnels between AWS and a simulated on prem environment.
2. Read up on Strongswan suite and RRR which are opensource VPN and BGP solutions respectively. This should come in handy in real projects.
3. Used Transit Gateway with attachments. Used BGP to dynamically exchange routes between the customer router and the Transit gateway.
4. We have true High availability with 2 pairs of tunnels between the customer and AWS.

Mistakes, gotchas

1. I had to wait a long time to get the IPSEC tunnels up. I started wondering if I made a mistake, Eventually they came up.
2. It took a long time for BGP to propagate. I started wondering if I should tear it down and redo it but it propagated eventually.

TODO?

1. More networking and DNS projects.