Giter VIP home page Giter VIP logo

terraform-aws-vpc's Issues

Please define the scope and goals of the AWS-IA Modules vs Terraform AWS Modules

Hi AWS Integration and Automation πŸ‘‹

There are several concerns over the modules being developed under AWS-IA vs Terraform AWS Modules

https://www.youtube.com/watch?v=h21sd7-hQoc&t=911s
https://registry.terraform.io/namespaces/terraform-aws-modules
https://twitter.com/andrewbrown/status/1442915716177350657

Due to:

  • a lack of understanding of the scope and goals of AWS-IA modules
  • the alpha state of the AWS-IA modules
  • the marketing rollout of the alpha modules
  • the replacement in AWS Samples and the purpose to serve them via AWS QuickStarts

It's drawing lots of speculation in the AWS Terraform community and there is fear of negative impact to the community and reasons why.

Define the Scope

My recommendation to AWS is to clearly define the goals of these modules and how they will be different from Terraform AWS Modules. Here are reasons why you want to develop your own modules:

  • The open-source Licensing of Terraform AWS Modules does not meet the legal expectations for AWS or its customers and serves to cause friction for the adoption of AWS and Terraform for specific organizations, and so these alternative modules are provided to alleviate that issue.
  • AWS wishes to provide AWS modules with more robust testing, and these modules are being written to adopt TerraTest end-to-end integration tests
  • AWS wants to provide simple sample modules not intended recommended for enterprise production use cases but rather a base template that customers can fork and adapt to meet their organization's requirements. AWS wants to set a good baseline to put customers on the right path but these AWS modules are limited compares to Terraform AWS Modules.
  • AWS wants to provide the benefit of technical support to a suite of AWS modules for their customers and has created their own modules so they can control the speed at which they can roll out bug fixes to meet SLAs. AWS-IA modules are limited compared to Terraform AWS Modules but that's the trade.

Make Effort to Work with the Community

AWS does not have to but it's appreciated to include community members in their OSS efforts who have grown the AWS Terraform space for the last 5 years to see if there is inclusion or collaboration or at least acknowledge the existence of these other projects to date.

AWS track record with OSS projects has been spotty, we (the community) understand AWS is a company with self-interests and sometimes they end up eating other people's lunch as of course of business, no different than the whale sifting the sea of phytoplankton.

https://news.ycombinator.com/item?id=24802924

It's just how you go about it.

β€” Andrew Brown (AWS Community Hero)

For the private subnet connect_to_public_natgw = false or connect_to_public_natgw = null is treated as connect_to_public_natgw = true

For the private subnet
connect_to_public_natgw = false
or
connect_to_public_natgw = null
or
connect_to_public_natgw = true

has same result. The route to NATGW is created.

Only way to prevent route to NATGW is to remove the key
#connect_to_public_natgw = false

The expectation is that for connect_to_public_natgw = false or connect_to_public_natgw = null the route to NATGW is not created

Region not declared in deploy

β”‚ Error: Reference to undeclared input variable
β”‚
β”‚ on main.tf line 14, in provider "aws":
β”‚ 14: region = var.region
β”‚
β”‚ An input variable with the name "region" has not been declared. This
β”‚ variable can be declared with a variable "region" {} block.

support cloudwan core network subnets

module "vpc" {
  source  = "aws-ia/vpc/aws"
  version = ">= 1.0.0"

  name       = "tgw"
  cidr_block = "10.0.0.0/16"
  az_count   = 2

  subnets = {
    public = {
      netmask                   = 24
      nat_gateway_configuration = "single_az"
      route_to_core_network  = ["10.0.0.0/8"]
    }

    private = {
      netmask                  = 24
      route_to_nat             = true
      route_to_core_network = ["10.0.0.0/8"]
    }

    core_network = {
      netmask                = 28
      core_network_id  =  awscc_networkmanager_core_network.example.id
      route_to_nat         = false
      ipv6_support        = true
    }
  }
}

tags specified at the subnet level has no effect

I am expecting that tags specified at each subnet level are applied to the specified subnets, merged with the tags specified at the VPC level.
The outcome is that only VPC level tags are applied. Subnet level tags are ignored.

[Enhancement] Export route table associations

Problem statement:
I have a resource (RDS custom for Oracle) in a private subnet that needs to communicate with S3 via a Gateway endpoint. During the termination of the instance, the instance will pull scripts from S3 to execute on the instance prior to shutting down. When operating terraform destroy, the Terraform engine tears down the route table associations early in the lifecycle, and the RDS instance is not able to connect to the S3 Gateway endpoint.

To be able to explicitly add a depends_on to the route table association, I would like to request the route table associations be exported.

I also welcome other suggestions regarding the route table associations.

2.x Breaking Changes

Changes from 1.x to 2.x

Features & Enhancements

  • Ability to create arbitrary amounts of private subnets. Previously was only capable of 3 types: public, private, transit gateway. The terms public and transit_gateway are reserved keywords for those subnet types and all other keys used in var.subnets.<> are assumed to be type private.
  • Many private subnet related resources had to be renamed. Most changes are accomplished programatically using a moved blocks but some require manual terraform state mv commands. see below.
  • route_to_nat has been changed to connect_to_public_natgw to clarify the nat is in the public subnet & to diverge from the route_to nomenclature which expects a route destination like input.
  • Can pass cidr or prefix list id to route_to_transit_gateway argument. Previously was a list of CIDRs that could only accept 1 item.
  • Many changes to Outputs available. Removed outputs marked as deprecated, separated grouped subnet attribute outputs into 3 public_, tgw_, and private_. Since you can have several private subnet declarations we group based on the name scheme <your_key_name>/az.

Bugs

  • Fixed a bug where VPCs that were built with a CIDR from IPAM were not idempotent between terraform runs

For help upgrading see our upgrading guide

aws_route involving TGW should wait until TGW is attached to VPC

I found an issue with terraform-aws-vpc module.
When using TGW
resource aws_route.public_to_tgw and resource aws_route.private_to_tgw may sometime fail with error that tgw does not exist.

I suspect it happens because
resource aws_ec2_transit_gateway_vpc_attachment.tgw and resource aws_ec2_transit_gateway_route_table_association.tgw are not yet complete.

I think, if aws_routes in question wait for these the TGW VPC attachments and TGW route table association, these errors can be avoided

Tags for every resource show needing updates every time I run apply

So even if I do not make any changes to terraform code and expect to see 0 diffs I get this for every resource created by the module.
I must be doing something wrong because I'm sure someone would have complained by now, but can't seem to figure it out.
It looks like the Key vs Value pairs just get rotated around in the list of tags.
image

Here is how I pass common tags in:

module "vpc" {
  source  = "registry.terraform.io/aws-ia/vpc/aws"
  version = "= 3.2.1"

  name = "${var.name}-vpc"
  tags = local.common_tags

  cidr_block = var.cidr_block
  az_count   = var.az_count
  ...
}

And here is how they are defined (merged with top level tags):

locals {
  common_tags = merge(var.common_tags, {
    environment = var.name
  })
}

and the top level module passes these tags in:

locals {
  common_tags = {
    managed_by        = "terraform"
    terraform_project = "core_infra"
  }
}

Any help is appreciated here.

Unsupported Attribute in "nat_eip_#".

Error: Unsupported attribute
β”‚
β”‚ on outputs.tf line 22, in output "nat_eip_3":
β”‚ 22: value = module.aws-ia_vpc.nat_eip_3
β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ β”‚ module.aws-ia_vpc is a object, known only after apply
β”‚
β”‚ This object does not have an attribute named "nat_eip_3".

VPC not found when using ipam

I am trying to create a VPC with IPAM:

module "vpc" {
  source  = "aws-ia/vpc/aws"
  version = ">= 1.0.0"

  name     = var.network_map["example"].suffix
  az_count = 2

  vpc_ipv4_ipam_pool_id   = module.create_ipam.pools_level_2[var.network_map["example"].ipam_region].id
  vpc_ipv4_netmask_length = var.network_map["example"].netmask_length

  subnets = {
    public = {
      netmask                   = 24
      nat_gateway_configuration = "all_azs"
    }
    private = {
      netmask      = 24
      route_to_nat = true
    }
  }
}

And the first pass would fail with the following error:
β”‚ Error: AWS SDK Go Service Operation Incomplete
β”‚
β”‚ with module.vpc.awscc_ec2_route_table.public["item"],
β”‚ on .terraform/modules/vpc/main.tf line 47, in resource "awscc_ec2_route_table" "public":
β”‚ 47: resource "awscc_ec2_route_table" "public" {
β”‚ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: The
β”‚ 'vpc-XXXXXXXXXXXXXXXXXXX' does not exist (Service: Ec2, Status Code: 400, Request ID: ....

When I tried to rerun it (making no changes) the script would try to destroy the VPC which would fail because of the dependencies. It was trying to destroy the VPC because the first pass already created a VPC which assigned a cidr range and the 2nd pass would have a difference cidr range. I tried to resolve the initial failure by replacing all the vpc_id calls to the following, but it didn't do any good:

vpc_id             = local.create_vpc ? aws_vpc.main[0].id : local.vpc.id

I also tried putting depends-on statements for various resources thinking it might just need time, but that didn't help. It keeps throwing the same error even when its done toward the end of execution. I even put a data command in to check the VPC existed and that works without issue. The VPC does exist when I go look at it in the console so I'm unsure how to resolve this.

Experiment "module_variable_optional_attrs" is no longer available.

Hello!
Getting below error when trying to use the module from TF v1.3.2
Github Issue

module "vpc" {
  source  = "aws-ia/vpc/aws"
  version = "3.0.0"

  name       = "vpc"
  cidr_block = "10.0.0.0/20"
  az_count   = 3

  subnets = {
    public = {
      netmask                   = 24
      nat_gateway_configuration = "all_azs" # options: "single_az", "none"
    }

    private = {
      netmask                 = 24
      connect_to_public_natgw = true
    }
  }

  vpc_flow_logs = {
    log_destination_type = "cloud-watch-logs"
    retention_in_days    = 180
    traffic_type         = "REJECT"
  }
}

image

AWS provider version constraints require bump to support newer resources

The current version constraint for the AWS provider in the module is set to >=3.73.0 however the module uses resources that do not exist in this version of the provider specifically aws_networkmanager_vpc_attachment and aws_networkmanager_attachment_accepter. These resources were added in v4.27.0.

If the resolved/matched version of the provider in a child module is below v4.27 the following is thrown:

β”‚ Error: Invalid resource type
β”‚
β”‚   on .terraform\modules\vpc\main.tf line 351, in resource "aws_networkmanager_vpc_attachment" "cwan":
β”‚  351: resource "aws_networkmanager_vpc_attachment" "cwan" {
β”‚
β”‚ The provider hashicorp/aws does not support resource type "aws_networkmanager_vpc_attachment".
β•΅
β•·
β”‚ Error: Invalid resource type
β”‚
β”‚   on .terraform\modules\vpc\main.tf line 369, in resource "aws_networkmanager_attachment_accepter" "cwan":
β”‚  369: resource "aws_networkmanager_attachment_accepter" "cwan" {
β”‚
β”‚ The provider hashicorp/aws does not support resource type "aws_networkmanager_attachment_accepter".

Further the submodule modules/flow_logs/modules/s3_log_bucket uses the aws_s3_bucket_server_side_encryption_configuration and aws_s3_bucket_lifecycle_configuration resources introduced in v4.0.0 however the version constraint is set to ">= 3.72.0"

 Error: Invalid resource type
β”‚
β”‚   on .terraform\modules\vpc\modules\flow_logs\modules\s3_log_bucket\main.tf line 17, in resource "aws_s3_bucket_server_side_encryption_configuration" "flow_logs":
β”‚   17: resource "aws_s3_bucket_server_side_encryption_configuration" "flow_logs" {
β”‚
β”‚ The provider hashicorp/aws does not support resource type "aws_s3_bucket_server_side_encryption_configuration".
β•΅
β•·
β”‚ Error: Invalid resource type
β”‚
β”‚   on .terraform\modules\vpc\modules\flow_logs\modules\s3_log_bucket\main.tf line 27, in resource "aws_s3_bucket_lifecycle_configuration" "flow_logs":
β”‚   27: resource "aws_s3_bucket_lifecycle_configuration" "flow_logs" {
β”‚
β”‚ The provider hashicorp/aws does not support resource type "aws_s3_bucket_lifecycle_configuration".

include some examples for data abstraction from outputs

It's not always immediately obvious how to get specific data out of outputs. for example: how to get a list of subnet_ids

example configuration:

module "vpc" {
  source  = "aws-ia/vpc/aws"
  version = ">= 1.0.0"

  name       = "multi-az-vpc"
  cidr_block = "10.0.0.0/20"
  az_count   = 3

  subnets = {
    private = { netmask = 24 }
  }
}

list of private subnet_ids

> [ for _, value in module.vpc.private_subnet_attributes_by_az: value.id]
[
  "subnet-04a86315c4839b519",
  "subnet-02a7249c8652a7136",
  "subnet-09af79b5329b3681f",
]

map of subnet_ids by az

> { for key, value in module.vpc.private_subnet_attributes_by_az: key => value.id}
{
  "us-east-1a" = "subnet-04a86315c4839b519"
  "us-east-1b" = "subnet-02a7249c8652a7136"
  "us-east-1c" = "subnet-09af79b5329b3681f"
}

Support for enabling vpc flow logs

We should have an optional argument to enable VPC flow logs. To enable flow logs we will need to take a log group name and iam role as inputs, or create them dynamically.

Possible bug with tags module

We are seeing an error recently. Our use of the module looks like this:

module "vpc" {
  source  = "aws-ia/vpc/aws"
  version = ">= 1.0.0"
...

And we're seeing this now in the terraform output:

Error: Unsupported argument

  on .terraform/modules/vpc/data.tf line 68, in module "tags":
  68:   tags = var.tags

An argument named "tags" is not expected here.

Hmm. Any ideas? Thanks!

Module not rerunable with ipam

When I call the modules to create the VPC and pass it IPAM and rerun the terraform, because the CIDR assigned by ipam changes every time you run the terraform. This appears to be caused by the data call:

data "aws_vpc_ipam_preview_next_cidr" "main" {
  count = var.vpc_ipv4_ipam_pool_id == null ? 0 : 1

  ipam_pool_id   = var.vpc_ipv4_ipam_pool_id
  netmask_length = var.vpc_ipv4_netmask_length
}

If you rerun the module after creating a VPC, it gets a new CIDR and tries to destroy the VPC and all resources because the VPC cidr "changed".

example (run the ipam first and then uncomment the VPC module call and run it twice more)

variable "network_map" {
  description = "map of network modules, check variable details on modules/network"
  type        = map(any)
  default = {
    "ci" = {
      suffix = "ci_vpc"
      ipam_region = "us-west-2/non_prod"
      netmask_length = 22
    },
  }
}

module "create_ipam" {
  source   = "github.com/aws-ia/terraform-aws-ipam"
  
  top_cidr = ["10.0.0.0/8"]
  top_name = "Global ipam"

  pool_configurations = {
    us-west-2 = {
      description = "2nd level, locale us-west-2 pool"
      cidr        = ["10.0.0.0/14"]
      locale = "us-west-2"

      sub_pools = {
        non_prod = {
          name                 = "non_prod_ipam"
          cidr                 = ["10.0.0.0/16"]
        }

        prod = {
          name                 = "prod_ipam"
          cidr                 = ["10.1.0.0/16"]
        }
      }
    },
  }
}

module "vpc" {
  source  = "aws-ia/vpc/aws"
  version = ">= 1.4.1"

  for_each = var.network_map

  name     = each.value.suffix
  az_count = 2

  vpc_ipv4_ipam_pool_id   = module.create_ipam.pools_level_2[each.value.ipam_region].id
  vpc_ipv4_netmask_length = each.value.netmask_length

  subnets = {
    public = {
      netmask                   = 24
      nat_gateway_configuration = "all_azs"
    }
    private = {
      netmask      = 24
      route_to_nat = true
    }
  }
}

additional problematic behavior

Because it uses the data command, if a ipam pool is "full" even the destroy command will fail because it tries to acquire a new ipam and it throws that there isn't enough room in ipam given the netmask.

The data command also becomes an issue when you try to spool ipam and VPC in the same module (for the multi-account setup I'm building with the ipam and VPC stuff in a central "networking" account). It throws the following error:

β”‚ Error: Invalid count argument
β”‚
β”‚   on .terraform/modules/vpc/data.tf line 41, in data "aws_vpc_ipam_preview_next_cidr" "main":
β”‚   41:   count = var.vpc_ipv4_ipam_pool_id == null ? 0 : 1
β”‚
β”‚ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to first apply only the resources that the count depends on.

[Bug]Enable Local zones breaks network topology

Issue Details:

Enabled the AKL local zone for Sydney region, after re-run a terraform plan, it’s going to recreate the resources in the new subnets and all resources.

Also local zones can only have limit services, no TGW support, no Nat gateway support, it will break the desired network configuration and topology.

Further investigation:

calculate_subnets module are using data resources to fetch from AWS.

so before data.aws_availability_zones.current returns:

    β€œap-southeast-2a”,
    β€œap-southeast-2b”,
    β€œap-southeast-2c”,

but post-enabled local zones:
it returns:

    β€œap-southeast-2-akl-1a”,
    β€œap-southeast-2a”,
    β€œap-southeast-2b”,
    β€œap-southeast-2c”,

the current logics slice the first x (based on the az_count)

so it and destroy and create resources for those azs.

β€œap-southeast-2-akl-1a”,
β€œap-southeast-2a”,
β€œap-southeast-2b”,

Suggest add aws-ia/vpc/aws to allow explicit specify configuration for az for VPC

module "vpc" {
  source  = "aws-ia/vpc/aws"
  version = "= 4.2.1"

  name       = "demo-vpc"
  cidr_block = "10.0.0.0/20"
  az_count   = 3
  azs = ["ap-southeast-2a", "ap-southeast-2b", "ap-southeast-2c"]

To prevent the similar issues in the future.

`aws_vpc_ipv4_cidr_block_association`

  • ability to create secondary cidr blocks
  • ability to create public/private subnets for the secondary cidrs
  • will only handle routing for igw & nat gateways

validation bug

β”‚ Error: Invalid variable validation result
β”‚ 
β”‚   on .terraform/modules/vpcs/variables.tf line 197, in variable "subnets":
β”‚  197:     condition     = try(var.subnets.private.route_to_nat, false) ? try(var.subnets.private.route_to_transit_gateway[0] != "0.0.0.0/0", true) : null
β”‚     β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚     β”‚ var.subnets.private is object with 3 attributes
β”‚     β”‚ var.subnets.private.route_to_nat is false
β”‚ 
β”‚ Validation condition expression must return either true or false, not null.

Explore implementing Local Zone support

its possible we could allow users to pass a list of local-zone ids instead of a az_count. I can think of 2 primary considerations:

  1. updating local.azs to be a list of LZs if provided

Subnets should be a generic map of subnets

Proposed idea: do not enforce naming conventions on subnet types (private, public, transit_gateway, etc). This will allow users to create arbitrary subnet amounts. For example, currently, users can only create 1 grouping of private subnets.

Idea 1: create abstract module concepts for each and allow users to specify in the map itself:

Pros/Cons:

  • - breaking change
  • + allows for defining subnet types in modules that are easier to understand
subnets = {
  myprivate = {
    type         = "private"
    netmask      = 24
    route_to_nat = "publicsubnets"
  }

  publicsubnets = {
    type                      = "public"
    netmask                   = 24
    nat_gateway_configuration = "all_azs" # options: "single_az", "none"
  }
}

Idea 2: create generic subnet module and allow any variable to be passed:

Pros/Cons:

  • + likely non breaking change
  • - code inside new module subnet would be complex
subnets = {
  myprivate = {
    netmask      = 24
    route_to_nat = "publicsubnets"
    routes = [{
       subnet  = "tgw"
       cidr    = "10.0.0.0/8"
    },
    {
       subnet = "nat"
       cidr   = "0.0.0.0/0"
  }]]
  }

  publicsubnets = {
    type     = "public"
    netmask  = 24
    nat_gateway_configuration = "all_azs" # options: "single_az", "none"
    routes = [{
       subnet = "tgw"
       cidr   = "10.0.0.0/8"
    },
    {
       subnet = "igw"
       cidr   = "0.0.0.0/0"
    }]
}

idea 2 open questions:

  • how to signify that a route should go to an appliance (igw, nat)?

Replace awscc resources with aws resources

Replace these resources:

  • awscc_ec2_route_table_association
  • awscc_ec2_route_table

Make sure tags are still working

Determine if we should remove calls to the label module

Optionally route to TGW from public/private subnets

updates have been requested:

  • route from public/private to tgw
  • allow tgw to route to nat
  subnets = {
    public = {
      netmask = 24
      # "0.0.0.0/0" is explicitly blocked for map to tgw
      route_to_transit_gateway = ["10.0.0.0/8"]
    }

    private = {
      netmask = 24
      # if route_to_nat = true
      # route_to_transit_gateway != 0.0.0.0/0
      route_to_nat = true
      route_to_transit_gateway = ["0.0.0.0/0"]
    }

    transit_gateway = {
      netmask            = 24
      transit_gateway_id = aws_ec2_transit_gateway.example.id
      route_to_nat       = true # default false
      transit_gateway_default_route_table_association = true
      transit_gateway_default_route_table_propagation = true
    }
  }

Support to enable "appliance_mode_support" in the TGW attachment

For some type of VPCs (Inspection VPCs), appliance mode is needed to not drop packets when the response traffic comes from a different AZ. By default, this value is "disable", so having the possibility to change it when defining the transit_gateway subnets will enable the configuration.
Proposed add-on in transit_gateway subnets:

subnets = {
  transit_gateway = {
    netmask                                         = 24
    transit_gateway_id                              = aws_ec2_transit_gateway.example.id
    route_to_nat                                    = false
    transit_gateway_default_route_table_association = true
    transit_gateway_default_route_table_propagation = true
    appliance_mode_support                          = "enable"
  }
}

Add VPC CREATE

Adding the following in main.tf with the variable defined in variables.tf will allow us to create the VPC conditionally similar to while calling from deploy directory.

create_vpc = var.create_vpc

TGW route back to tgw

    transit_gateway = {
      netmask                                         = 28
      transit_gateway_id                              = aws_ec2_transit_gateway.example.id
      route_to_nat                                    = false
      transit_gateway_default_route_table_association = true
      transit_gateway_default_route_table_propagation = true
     route_to_tgw = [""]
    }

(association to rt)

secondary cidr feature breaks if connect_to_public_natgw = true

If I create secondary cidr as below

module "secondary" {
  source  = "aws-ia/vpc/aws"
  version = ">= 2.0.0"

  name       = "secondary-cidr"
  cidr_block = "10.2.0.0/16"

  vpc_secondary_cidr = true
  vpc_id             = module.vpc.vpc_attributes.id
  az_count           = 2

  subnets = {
    eks = {
      name_prefix             = "vpce"
      cidrs                   = ["10.2.0.0/18", "10.2.64.0/18"]
      connect_to_public_natgw = true
      #route_to_transit_gateway = "pl-"
      #tags = var.vpc_shared_svc_vpce_subnet_tags
    }
  }
}

it breaks with following error

   on .terraform\modules\secondary\main.tf line 172, in resource "aws_route" "private_to_nat":
β”‚  172:   nat_gateway_id = try(aws_nat_gateway.main[split("/", each.key)[1]].id, aws_nat_gateway.main[local.nat_configuration[0]].id)
β”‚     β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚     β”‚ aws_nat_gateway.main is object with no attributes
β”‚     β”‚ each.key is "eks/us-east-1a"
β”‚     β”‚ local.nat_configuration is empty tuple
β”‚
β”‚ Call to function "try" failed: no expression succeeded:
β”‚ - Invalid index (at .terraform\modules\secondary\main.tf:172,44-69)
β”‚   The given key does not identify an element in this collection value.
β”‚ - Invalid index (at .terraform\modules\secondary\main.tf:172,118-121)
β”‚   The given key does not identify an element in this collection value: the collection has no elements.
β”‚
β”‚ At least one expression must produce a successful result.

Because aws_nat_gateway.main is not available in module.secondary.
without connect_to_public_natgw = true, it works but route to NATGW is missing in the subnet route tables of the secondary

Cannot route to internet through transit gateway

Attempting to setup a multaccount transit network where the spoke account can route through the transit gateway and get to the internet. Created two new fresh aws accounts to apply this terraform and let it setup all of the attachments, route tables, subnets,etc. but I can't seem to get to the internet from the spoke. The instance sitting in the spoke can ping things in the network account but can't get to the internet.

                  - Core Network Account -                                                              - Spoke Account -
<igw>--<publicsubnet>--<natgw>--<privatesubnet>--<tgw_attachment>--<tgw>--<tgw_attachment>--<privatesubnet>--<instance>

main.tf

resource "aws_ec2_transit_gateway" "network_tgw" {
  provider                       = aws.core_prod_network
  amazon_side_asn                = 65412
  auto_accept_shared_attachments = "enable"
  dns_support                    = "enable"
  description                    = "Core Prod Network Transit Gateway"
}

resource "aws_ram_resource_share" "core_prod_network" {
  provider                  = aws.core_prod_network
  name                      = "Transit Gateway Resource Share"
  allow_external_principals = true
  tags = {
    Name = "core-prod-network-tgw-resource-share"
  }
}

# Share the transit gateway...
resource "aws_ram_resource_association" "core_prod_network" {
  provider           = aws.core_prod_network
  resource_arn       = aws_ec2_transit_gateway.network_tgw.arn
  resource_share_arn = aws_ram_resource_share.core_prod_network.id
}

#### For every Spoke Account, add this to attach tgw
## ..with the core_prod_devops account
resource "aws_ram_principal_association" "core_prod_devops" {
  provider           = aws.core_prod_network
  principal          = var.vega_accounts.core_prod_devops.id
  resource_share_arn = aws_ram_resource_share.core_prod_network.id
}

## core_prod_network
module "core_prod_network_vpc" {
  providers = {
    aws   = aws.core_prod_network
    awscc = awscc.core_prod_network
  }
  source     = "aws-ia/vpc/aws"
  name       = "core-prod-network-vpc"
  cidr_block = "10.3.0.0/16"
  az_count   = 2

  subnets = {
    public = {
      name_prefix               = "core-prod-network-public"
      netmask                   = 24
      nat_gateway_configuration = "all_azs"
      route_to_transit_gateway  = ["10.0.0.0/8"]
    }

    private = {
      name_prefix              = "core-prod-network-private"
      netmask                  = 24
      route_to_nat             = true
      route_to_transit_gateway = ["10.0.0.0/8"]
    }

    transit_gateway = {
      name_prefix                                     = "core-prod-network-transit"
      netmask                                         = 28
      transit_gateway_id                              = aws_ec2_transit_gateway.network_tgw.id
      route_to_nat                                    = false
      transit_gateway_default_route_table_association = true
      transit_gateway_default_route_table_propagation = true
    }
  }
}


### Output Block we need after every VPC creation
output "core_prod_network_vpc_id" {
  value = module.core_prod_network_vpc.vpc_attributes.id
}

output "core_prod_network_vpc_info" {
  value = module.core_prod_network_vpc
}

## core_prod_devops
module "core_prod_devops_vpc" {
  providers = {
    aws   = aws.core_prod_devops
    awscc = awscc.core_prod_devops
  }
  source     = "aws-ia/vpc/aws"
  name       = "core-prod-devops-vpc"
  cidr_block = "10.7.0.0/16"
  az_count   = 2

  subnets = {
    private = {
      name_prefix              = "core-prod-devops-private"
      netmask                  = 24
      route_to_nat             = false
      route_to_transit_gateway = ["0.0.0.0/0"]
    }

    transit_gateway = {
      name_prefix                                     = "core-prod-devops-transit"
      netmask                                         = 28
      transit_gateway_id                              = aws_ec2_transit_gateway.network_tgw.id
      route_to_nat                                    = false
      transit_gateway_default_route_table_association = true
      transit_gateway_default_route_table_propagation = true
    }
  }
}

I don't know if this is something that you guys have to add to the routing tables in the terraform to account for this use case but something is missing from routing.

For the public subnet nat_gateway_configuration = "none" is broken

For the public subnet
nat_gateway_configuration = "none"
or
nat_gateway_configuration = null
or
#nat_gateway_configuration = "none"
Gives following error

Error: Error in function call
β”‚
β”‚   on .terraform\modules\shared_services_vpc\data.tf line 53, in locals:
β”‚   53:     { for az in local.azs : az => { id : try(aws_nat_gateway.main[az].id, aws_nat_gateway.main[local.nat_configuration[0]].id) } }
β”‚     β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚     β”‚ aws_nat_gateway.main is object with no attributes
β”‚     β”‚ local.nat_configuration is empty tuple
β”‚
β”‚ Call to function "try" failed: no expression succeeded:
β”‚ - Invalid index (at .terraform\modules\shared_services_vpc\data.tf:53,66-70)
β”‚   The given key does not identify an element in this collection value.
β”‚ - Invalid index (at .terraform\modules\shared_services_vpc\data.tf:53,119-122)
β”‚   The given key does not identify an element in this collection value: the collection has no elements.
β”‚
β”‚ At least one expression must produce a successful result.

for the above conditions local.nat_configuration is [] therefore local.nat_configuration[0] fails.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.