AzureInterRegionalRoutingLab

The goal of this repo is to showcase a couple of InterRegion routing usecases.

Goals

Create a lab environment that can be used for demonstrating cross region traffic flow.

Prerequisites

Terraform
Terragrunt
Azure Account with RG and SA

Terragrunt

applying whole region or subscription

This should be done only the first time around. Once the whole infra is deployed future changes are to be applied single module at a time

cd terragrunt/subscription/eu-west-1
terragrunt run-all apply

applying single module

cd terragrunt/subscription/us-east-2/az-fw/terragrunt.hcl
terragrunt apply

Terragrunt folder structure

terragrunt (root folder for TG. Contains a config file and one or more "subscriptions")
├── terragrunt.hcl (Defines providers and backends)
└── subscription (subscriptions contain a config file and one or more "regions")
    ├── subscription.hcl (subscription level vars)
    ├── eu-west-1 (regions contain individual terraform module deployments)
    │   ├── region.hcl (region level vars)
    │   ├── az-fw
    │   │   └── terragrunt.hcl (terraform deployment vars and dependencies on other deployments)
    │   └── hub-and-spoke
    │       └── terragrunt.hcl
    └── us-east-2
        ├── region.hcl
        ├── az-fw
        │   └── terragrunt.hcl
        └── hub-and-spoke
            └── terragrunt.hcl

Azure

Private endpoints finallly have NSG and UDR support https://azure.microsoft.com/en-us/updates/general-availability-of-network-security-groups-support-for-private-endpoints/ https://azure.microsoft.com/en-us/updates/general-availability-of-user-defined-routes-support-for-private-endpoints/

Unfortunatelly, terraform configuration renamed some of the api properties to make them more confusing: hashicorp/terraform-provider-azurerm#6334

enforce_private_link_endpoint_network_policies needs to be set to false in order for the provider to set PrivateEndpointNetworkPolicies: Enabled NOTE: This will be fixed in version 4 of azurerm provider

How PrivateEndpointNetworkPolicies: Enabled works

When this setting is enabled on a subnet. All private endpoints in that subnet will have their routes potentially invalidated in that AND all other subnets (if the right override rule exists).

In case a subnet has this setting enabled and /32 has been injected from a PE in a peered subnet that doesn't have it enabled. The route will NOT override. This could be phrased as: PrivateEndpointNetworkPolicies affects only the private endpoint /32 routes that are created from private endpoints coming from subnets with this setting enabled.

for context on the above screenshot:

10.0.0.0/16 - this vnet has PrivateEndpointNetworkPolicies: Enabled (false passed to terraform)
10.1.0.0/16 - this vnet has PrivateEndpointNetworkPolicies: Disabled (true passed to terraform)

On the screenshot above we can see that only one InterfaceEndpoint has been invalidated (very bottom of the list). Notice that routes exist for overriding both of them, but only one is working. This is because the setting affects all private endpoint interface routes (in whichever vnet they are created) of private endpoint that exist in the subnet, but doesn't work on private endpoint interface routes from other subnets.

Understanding the need to SNAT for HUB->PE traffic.

In case you are using private endpoints and want to ensure the traffic is inspected by the azure firewall you are probably leveraging the new PrivateEndpointNetworkPolicies: Enabled feature. There are certain caveats when using Private Endpoints with this feature and trying to inspect the Traffic with the AzFirewall. For traffic originating in a Spoke Vnet and flowing to a PrivateEndpoint in a spoke VNet there are no special considerations. If the UDRs are properly setup, the traffic should flow without issues. However, for hub to private endpoint traffic there are special considerations. SNAT is needed in this case: