What you will learn
- Site-to-Site VPN — Virtual Private Gateway, Customer Gateway, and route propagation
- VPN CloudHub — connecting multiple branch offices using one Virtual Private Gateway
- Direct Connect — Private VIF vs Public VIF, DX Gateway for multi-region access
- Why Direct Connect alone is not encrypted and what to do about it
- Direct Connect resiliency levels — the four tiers from basic to maximum
- AWS Transit Gateway — hub-and-spoke replacing mesh VPC peering
- ECMP and bandwidth multiplication through multiple VPN tunnels to Transit Gateway
- VPC Traffic Mirroring — capture live network traffic for security analysis
- IPv6 on AWS and the Egress-Only Internet Gateway for outbound-only IPv6
- AWS Network Firewall — stateful Layer 7 inspection inside your VPC
Why this matters
A large Indian enterprise — bank, insurance company, or government organisation — runs applications on-premises in their data centre and needs to extend into AWS. Simply exposing those applications to the public internet is not acceptable for compliance reasons. Site-to-Site VPN creates an encrypted private tunnel. But VPN over the public internet still has variable latency and packet loss. Direct Connect provides a dedicated private fibre connection with consistent latency — the right choice for latency-sensitive trading systems or high-volume data pipelines. As the company grows and creates 20 VPCs across accounts, Transit Gateway replaces a tangle of hundreds of individual peering connections with one central hub. Understanding these connectivity patterns is what separates an engineer who can design hybrid architectures from one who can only describe them.
Site-to-Site VPN
Site-to-Site VPN creates an encrypted tunnel over the public internet between your on-premises network and your AWS VPC. All traffic flows through this encrypted tunnel — as if the two networks were directly connected.
Two endpoints:
Virtual Private Gateway (VGW): AWS side of the tunnel Attached to your VPC One VPC → one VGW Customer Gateway (CGW): Your side of the tunnel A software resource in AWS representing your on-premises router or firewall You create it in AWS with the public IP of your on-premises deviceSetup flow:
On-premises router (public IP: 203.0.113.5) ↓ encrypted IPSec tunnel over public internetVirtual Private Gateway (attached to your VPC) ↓Your private VPC resourcesRoute Propagation:
Without route propagation you manually add routes to your VPC route table pointing on-premises CIDRs to the VGW. With Route Propagation enabled, the VGW automatically adds routes learned from the on-premises router via BGP.
## Create a Customer Gateway representing your on-premises routeraws ec2 create-customer-gateway \ --type ipsec.1 \ --public-ip 203.0.113.5 \ --bgp-asn 65000 \ --region ap-south-1 ## Create a Virtual Private Gateway and attach to VPCaws ec2 create-vpn-gateway \ --type ipsec.1 \ --region ap-south-1 aws ec2 attach-vpn-gateway \ --vpn-gateway-id vgw-0abc123456789def \ --vpc-id vpc-0abc123456789def \ --region ap-south-1 ## Create the VPN connectionaws ec2 create-vpn-connection \ --type ipsec.1 \ --customer-gateway-id cgw-0abc123456789def \ --vpn-gateway-id vgw-0abc123456789def \ --options StaticRoutesOnly=false \ --region ap-south-1 ## Enable route propagation on private route tableaws ec2 enable-vgw-route-propagation \ --route-table-id rtb-private-abc123 \ --gateway-id vgw-0abc123456789def \ --region ap-south-1RememberEach VPN connection consists of TWO IPSec tunnels for high availability. AWS provides two tunnel endpoints — configure both on your on-premises router so if one fails, traffic automatically flows through the other.
AWS VPN CloudHub
You have five branch offices — Delhi, Mumbai, Hyderabad, Pune, and Bangalore. Each needs to connect to AWS. Each also needs to connect to the other branches. Normally that would require connecting each branch to AWS and setting up meshed VPN tunnels between offices.
VPN CloudHub solves this with a hub-and-spoke model using one Virtual Private Gateway.
Delhi branch ──────┐Mumbai branch ──────┤Hyderabad branch ──────┼──> Virtual Private Gateway (hub)Pune branch ──────┤Bangalore branch ──────┘ Delhi to Hyderabad traffic:Delhi → VGW → Hyderabad (traffic routed through the hub automatically)All branches connect to the same VGW. AWS routes traffic between them. No direct branch-to-branch tunnels needed. Low cost, works over the public internet, encrypted via IPSec.
RememberVPN CloudHub traffic between branches travels through the VGW in AWS. It does NOT have access to resources inside the VPC unless explicitly configured. The VGW is the hub for branch connectivity — VPC access is separate.
AWS Direct Connect (DX)
Site-to-Site VPN uses the public internet — variable latency, packet loss possible, shared bandwidth. Direct Connect bypasses the public internet entirely with a dedicated physical fibre connection from your data centre to an AWS Direct Connect location.
Data centre → (private fibre) → DX Location → (AWS network) → Your VPC DX gives you: Consistent network performance — dedicated bandwidth, no sharing Reduced bandwidth costs for high-volume data transfers Private connectivity — traffic never touches the public internetDX bandwidth options: 1 Gbps, 10 Gbps, 100 Gbps. For smaller needs, partner connections from 50 Mbps.
SecurityDirect Connect is NOT encrypted by default. It is a private connection but the traffic is not encrypted. For data that requires encryption in transit (financial data, PII), run IPSec VPN over Direct Connect or use MACsec where supported.
Two types of Virtual Interfaces (VIFs):
Private VIF: Connects to one VPC via a VGW Access private resources inside the VPC (EC2, RDS, ElastiCache) Traffic stays on private IP addresses Public VIF: Connects to AWS public services (S3, DynamoDB, Glacier) Traffic uses public IP space but still travels on DX — not public internet Use when: downloading from S3 over DX without internet chargesDirect Connect Gateway — one DX connection, multiple VPCs and regions:
Without DX Gateway, a single DX connection reaches one VPC in one region via a Private VIF. To reach VPCs in different regions you need separate DX connections.
DX Gateway solves this:
On-premises data centre ↓ Single Direct Connect connectionDX Gateway (global, not region-specific) ├── VPC in ap-south-1 ├── VPC in ap-southeast-1 └── VPC in us-east-1One physical DX connection, access to VPCs worldwide through the DX Gateway.
Direct Connect Resiliency Levels:
| Level | Setup | Protects against |
|---|---|---|
| Non-redundant | One DX connection, one location | Nothing — single point of failure |
| High Resiliency | Two DX connections, two locations | Single connection or location failure |
| Maximum Resiliency | Four DX connections, two locations | Two simultaneous location failures |
| Backup VPN | DX primary + Site-to-Site VPN fallback | DX failure (VPN is the backup) |
RememberFor maximum resiliency you need DX connections at two geographically separate DX locations with two connections at each location — four connections total. This protects against any single location going offline.
AWS Transit Gateway
VPC Peering works for a small number of VPCs. You have 10 VPCs and every pair needs to communicate: that is 10×9/2 = 45 peering connections. You have 20 VPCs: 190 connections. Each peer must be managed individually with its own route tables.
Transit Gateway is a regional network transit hub. Every VPC and on-premises connection attaches to the TGW. Traffic between any two attachments routes through the TGW automatically.
Without Transit Gateway (10 VPCs, meshed peering): 45 individual VPC peering connections 45 pairs of route table entries Every new VPC requires connections to all 9 others With Transit Gateway (10 VPCs, hub-and-spoke): 10 attachments to TGW — one per VPC 10 route table entries New VPC: one new attachment, doneHow Transit Gateway routes:
VPC A (10.0.0.0/16) ──────┐VPC B (10.1.0.0/16) ──────┤VPC C (10.2.0.0/16) ──────┼──> Transit Gateway ──> (routes to all)On-prem (192.168.0.0) ──────┤VPN Connection ──────┘Key TGW facts:
Regional resource — VPCs in the same region attach directlyCross-region peering — TGW in ap-south-1 can peer with TGW in us-east-1Scales to thousands of VPCs and on-premises connectionsSupports IP Multicast (the only AWS service that does)Route tables on TGW control which attachments can reach whichECMP — Bandwidth Multiplication with Multiple VPNs:
ECMP (Equal Cost Multi-Path) routing allows traffic to flow over multiple paths simultaneously when they have equal routing cost.
Without ECMP: VPN tunnel 1 → 1.25 Gbps max (VPN tunnel limit) All traffic uses tunnel 1 → tunnel 2 is only a failover With TGW + ECMP + multiple VPN connections: VPN connection 1 (2 tunnels: 1.25 Gbps each) ──┐ VPN connection 2 (2 tunnels: 1.25 Gbps each) ──┤──> TGW (ECMP balances) VPN connection 3 (2 tunnels: 1.25 Gbps each) ──┘ Total effective bandwidth: up to 3 × 2.5 Gbps = 7.5 Gbps Add more VPN connections → multiply bandwidth furtherRememberVPC Peering only works through TGW — it does not support transitive routing. If VPC A peers with TGW and VPC B peers with TGW, A and B can talk through TGW. But if A peers directly with B and B peers directly with C, A cannot reach C through B — transitive peering is not supported and never was.
VPC Traffic Mirroring
Traffic Mirroring copies live network traffic from EC2 instances (ENIs) and sends it to a monitoring appliance for analysis — without affecting the original traffic.
Source: An ENI on a production EC2 instanceFilter: Which traffic to copy (by port, CIDR, protocol)Target: A Network Load Balancer or an ENI receiving the mirrored copyAppliance: Your security tool (IDS, Wireshark, packet analyser) receives the copy Production traffic: Request → EC2 → Response (unaffected, normal flow) ↓ (copy sent simultaneously) Monitoring Appliance (receives mirror, analyses patterns)Use cases:
Network security analysis and threat detectionTroubleshoot network connectivity issues with full packet captureDetect data exfiltration attempts by analysing outbound traffic patternsCompliance — full packet capture for financial regulationsIPv6 and Egress-Only Internet Gateway
IPv4 addresses are running out globally. IPv6 provides a vastly larger address space.
In AWS, IPv6 addresses are always public and internet-routable. There are no private IPv6 addresses in the way there are private IPv4 ranges (10.x.x.x, 192.168.x.x).
The problem with public IPv6 in private subnets:
Private subnets should be unreachable from the internet. With IPv4, NAT Gateway prevents inbound connections. With IPv6, there is no NAT — all addresses are public.
Egress-Only Internet Gateway:
Works like an Internet Gateway but for IPv6 — allows outbound IPv6 traffic from your VPC while blocking all inbound IPv6 connections from the internet.
Private IPv6 instance: Outbound: request → EIGW → internet → response back through EIGW → instance ✓ Inbound: internet → EIGW → BLOCKED ✗ EIGW for IPv6 = NAT Gateway for IPv4 (conceptually)Route table entry for EIGW:
Private route table with IPv6: Destination Target 10.0.0.0/16 local ::/0 eigw-abc123 ← all IPv6 traffic to EIGW (outbound only)AWS Network Firewall
A Security Group is a simple stateless (NACLs) or stateful (SGs) packet filter. WAF is Layer 7 for HTTP only. For full stateful Layer 7 inspection of ALL traffic inside your VPC — any protocol, not just HTTP — use AWS Network Firewall.
Network Firewall capabilities: Stateful packet inspection (tracks connection state) URL/domain filtering — block specific hostnames Protocol-based rules — allow only HTTPS, block FTP Intrusion Detection and Prevention (IDS/IPS) using Suricata rules AWS-managed rule groups updated automatically with threat intelligenceNetwork Firewall deployment:
Internet ↓Network Firewall (in its own subnet per AZ) ↓ (inspects and passes or drops)Application Load Balancer (in public subnets) ↓EC2 instances (in private subnets)Route tables direct traffic through the firewall subnet before it reaches the application. Traffic that does not pass inspection is dropped.
Network Firewall vs WAF vs Security Groups:
| Security Groups | NACLs | WAF | Network Firewall | |
|---|---|---|---|---|
| Layer | 4 (instance level) | 3-4 (subnet level) | 7 HTTP only | 3-7 all protocols |
| Stateful | Yes | No | Yes | Yes |
| URL filtering | No | No | Yes | Yes |
| IDS/IPS | No | No | No | Yes |
| Protocols | Any | Any | HTTP/HTTPS | Any |
RememberWAF = Layer 7 HTTP/HTTPS inspection for public-facing ALB, CloudFront, API Gateway. Network Firewall = stateful deep packet inspection for all traffic (any protocol) flowing between subnets inside your VPC. They solve different problems and can both be used together.
Hands-on Lab — Transit Gateway Connecting Two VPCs
Step 1 — Create two VPCs
VPC → Your VPCs → Create VPCName: vpc-a IPv4 CIDR: 10.1.0.0/16 Create Create again:Name: vpc-b IPv4 CIDR: 10.2.0.0/16 Create Create one private subnet in each:vpc-a: subnet 10.1.1.0/24 in ap-south-1avpc-b: subnet 10.2.1.0/24 in ap-south-1aStep 2 — Create a Transit Gateway
VPC → Transit Gateways → Create transit gatewayName: devops-tgwDefault route table association: EnableDefault route table propagation: EnableCreate transit gateway — wait until Available (2-3 minutes)Step 3 — Attach both VPCs to the Transit Gateway
VPC → Transit Gateway Attachments → Create attachmentTransit gateway: devops-tgwAttachment type: VPCVPC: vpc-a Subnet: 10.1.1.0/24Create attachment Create another:Transit gateway: devops-tgwVPC: vpc-b Subnet: 10.2.1.0/24Create attachment Wait until both show state: AvailableStep 4 — Update Route Tables
vpc-a private route table → Routes → Edit → Add route:Destination: 10.2.0.0/16 Target: devops-tgw vpc-b private route table → Routes → Edit → Add route:Destination: 10.1.0.0/16 Target: devops-tgw Now instances in vpc-a can reach instances in vpc-b using private IPs.Traffic flows: vpc-a instance → TGW → vpc-b instanceNo internet involved. Private routing throughout.Step 5 — Verify connectivity
Launch one EC2 in vpc-a subnet and one in vpc-b subnet.From vpc-a EC2, ping the private IP of vpc-b EC2.Without TGW: unreachable.With TGW: ping succeeds — two separate VPCs, private routing.Step 6 — Cleanup
Delete TGW Attachments (both) — wait until deletedDelete Transit GatewayTerminate EC2 instancesDelete subnets, route tables, then VPCsProduction Best Practices and Common Pitfalls
- Use Transit Gateway instead of VPC peering when you have more than 5 VPCs — peering does not scale, TGW does
- Run Site-to-Site VPN as backup for Direct Connect — DX alone without VPN fallback means a physical cable cut takes your hybrid connectivity offline
- Encrypt Direct Connect traffic with IPSec VPN over DX or MACsec — DX is private but not encrypted
- Configure both VPN tunnels on your on-premises router — AWS provides two tunnels per VPN connection, use both for HA
- Use ECMP with multiple VPN connections to Transit Gateway when you need more than 1.25 Gbps of VPN throughput
- Enable route propagation on private route tables — avoids manual route management as on-premises routes change
- Use Network Firewall for east-west traffic inspection between VPCs connected via TGW — Security Groups alone cannot inspect cross-VPC traffic at Layer 7
Quick Reference and Troubleshooting Commands
| Task | Command |
|---|---|
| List Transit Gateways | aws ec2 describe-transit-gateways --region ap-south-1 |
| List TGW attachments | aws ec2 describe-transit-gateway-vpc-attachments --region ap-south-1 |
| List TGW route tables | aws ec2 describe-transit-gateway-route-tables --region ap-south-1 |
| List VPN connections | aws ec2 describe-vpn-connections --region ap-south-1 |
| Check VPN tunnel status | aws ec2 describe-vpn-connections --query 'VpnConnections[*].VgwTelemetry' |
| List Direct Connect connections | aws directconnect describe-connections --region ap-south-1 |
| List DX Virtual Interfaces | aws directconnect describe-virtual-interfaces --region ap-south-1 |
| List Traffic Mirror sessions | aws ec2 describe-traffic-mirror-sessions --region ap-south-1 |
Common problems and fixes:
| Problem | Likely cause | Fix |
|---|---|---|
| VPN tunnel UP but traffic not routing | Route not added in VPC route table | Add route for on-premises CIDR pointing to VGW, enable route propagation |
| TGW attachment in pending state | Subnet not in the right AZ or insufficient permissions | Check subnet AZ matches TGW AZ support, verify IAM permissions |
| Cannot reach VPC B from VPC A via TGW | Missing route in one or both VPC route tables | Add routes in both VPCs pointing opposite CIDR to TGW |
| DX not providing expected latency improvement | Traffic still using internet path | Verify BGP routes prefer DX path, check on-premises router BGP config |
| IPv6 instances reachable from internet | Using IGW not EIGW for IPv6 | Replace 0::/0 → IGW route with 0::/0 → EIGW in private route table |
Common MistakeExpecting transitive routing through VPC peering. If VPC A peers with VPC B and VPC B peers with VPC C, traffic from A cannot reach C through B. VPC peering is not transitive. For transitive routing between multiple VPCs, use Transit Gateway — it was built specifically for this.
TipWhen you need both Direct Connect for normal operations and a cost-effective backup, configure Site-to-Site VPN as a failover. On your on-premises BGP router, assign a higher BGP metric (lower preference) to the VPN route and a lower metric (higher preference) to the DX route. Traffic uses DX normally. If DX fails, BGP automatically prefers the VPN route and traffic fails over — no manual intervention.
Common Mistakes to Avoid
Common MistakeTrying to peer VPCs with overlapping CIDR ranges. VPC Peering requires non-overlapping CIDRs — AWS cannot route between identical address spaces. Plan your IP ranges before creating any VPC. Changing a VPC CIDR after creation is not possible.
Common MistakeExpecting transitive routing through VPC Peering. If VPC A is peered with VPC B, and VPC B is peered with VPC C, A cannot reach C through B. Create a direct peering between A and C, or use Transit Gateway which handles hub-and-spoke routing natively.
TipFor connecting more than 3 VPCs, Transit Gateway is almost always the right choice over mesh VPC Peering. With 5 VPCs and full mesh peering you need 10 peering connections. With Transit Gateway you need 5 attachments and one TGW.
