[FEATURE] CIDR management capabilities, enabling at-scale spoke VPCs w/central routing about aws-secure-environment-accelerator HOT 2 CLOSED

IMPROVE VPC DEPLOYMENT FUNCTIONALITY

Enable inter-connected spoke VPCs, VPC templates, and user selectable VPCs

1. Add new internal search/replace variables called ${VPC_NAME}, ${OU_NAME} for config.json

• Only valid in vpc json object and resolves to vpc\name=value i.e. Dev
• Only valid in ou json object and resolves to organizational-units\value i.e. Dev

2. Create 3 new DDB tables: (bold fields are mandatory, blue required for lookup)

• table name: AccelPrefix-cidr-pools
  • fields: cidr, region, pool
  • cidr in format: 10.0.0.0/8
  • region in format: ca-cenral-1
  • pool: any string
• table name: AccelPrefix-cidr-vpc-assign
  • fields: status, region, cidr, pool, index, account-ou-key, vpc-name, account-id, vpc-id, date, requester
  • status: available || assigned || blocked
  • index: unique row identifier
  • date: date record last programmatically updated
  • account-ou-key: organizational-unit\value, workload-account-configs\value, mandatory-account-configs\value i.e. organizational-unit\Dev
  • account-ou-key and vpc-name are used for assignment and lookup
  • account-id and vpc-id are programmatically populated during vpc creation
    • reference only-not used for lookup
  • requester: “AcceleratorPrefix” || Manual || {userid}
• table name: AccelPrefix-cidr-subnet-assign
  • fields: status, region, cidr, vpc-index, account-ou-key, vpc-name, subnet-name, az, subnet-id, date, requester, sub-pool
  • sub-pool: any string
  • az: a single letter i.e. a, b, c

NOTE: Ensure all codified updates to these tables prevent concurrent updates to the same record

3. Add new config file variables inside all vpc objects (vpc\cidr-src and vpc\opt-in)

• opt-in: true || false, default value is “false” (current behaviour)

  • if opt-in is true, ignore current vpc creation steps
  • only valid in OU’s section of config file
  • will be used as part of part 6)

• cidr-src: “provided” || “lookup” || “dynamic”, default value is “provided”

  • valid in OU, mandatory and workload account sections of config file
  • both cidr and cidr2 must use the same src type
  • *“provided” - *config file as-is today
  • “lookup” – see 4) below
  • “dynamic” – see 5) below

4. cidr-src: “lookup”

   • gets vpc\cidr from cidr-vpc-assign DDB table
     • using status=assigned, region, account-ou-key, vpc-name
   • gets vpc\subnets\definitions\cidr from cidr-subnet-assign DDB table
     • using status=assigned, region, account-ou-key, vpc-name, subnet-name, az
   • the cidr values in the config file become optional (no longer used)
   • DDB tables are pre-populated manually by customer with values
   • SM validates all values exist in DDB table at early stage in SM, fails if error
   • when vpc created, vpc-id and subnet-id will be appended by Accelerator
     • account-id and vpc-id can be multi-value items (i.e. sandbox OU)
   • this will enable us to use the same sub-file to define each vpc in the ou/account sections in a multi-part config file, if vpc structures are identical
   • If a deployed cidr range changes in DDB, fail the SM

*5) *cidr-src: “dynamic”

• add new optional config file parameters in vpc objects
  • cidr-size parameter (and cidr2-size for vpc only, not subnets - see bullet 3/4)
    • beside: vpc\cidr and vpc\subnets\definitions\cidr
    • value: between 16 and 28
  • cidr-pool parameter (and cidr2-pool for vpc only, not subnets - see bullet 3/4)
    • beside: vpc\cidr and vpc\subnets\definitions\cidr
    • value: any string || the variable: ${OU_NAME}
    • ${OU_NAME} resolves to the value of the ou the vpc is being created in
  • plan for vpc\cidr2 to be an array, see GitHub PR670 (https://github.com/aws-samples/aws-secure-environment-accelerator/pull/670)
  • convert the subnet cidr2 value back to a cidr, at the subnet level we do not need to distinguish cidr1 from cidr2
• PROCESS:
  • For vpc\cidr, using cidr-vpc-assign
    • 1st: Check status=assigned, region, account-ou-key, vpc-name, pool (and use)
      • Ensure value has not changed since last SM execution, if deployed
    • 2nd: Check: status=available, region, pool, size
      • Change status=assigned, Set account-ou-key, vpc-name, date, requestor
      • (add account-id and vpc-id after vpc creation as a separate update to prevent deadlocks)
    • 3rd: Create a new block using NetAddr from cidr-pools table in cidr-vpc-assign
      • https://pypi.org/project/netaddr/
      • Sample code leveraging net-addr: https://github.com/aws-samples/aws-vpc-cidr-mgmt (use the ipmgmt.zip code as is, if desired)
      • Cannot overlap with any existing: status: available || assigned || blocked from cidr-vpc-assign for that region/pool
      • Divide the pool usig the first/smallest available space
      • Add the new cidr assignment to cidr-vpc-assign DDB table populating:
        • Status=assigned, region, cidr, pool, index=unique, account-ou-key, vpc-name, date, requester=”AccelPrefix”
  • For vpc\subnets\definitions\cidr, using cidr-subnet-assign
    • 1st: Check status=assigned, region, account-ou-key, vpc-name, subnet-name, az, vpc-index (and use)
      • Ensure value has not changed since last SM execution, if deployed
    • 2nd: Check: status=available, vpc-index, size, sub-pool
      • Change status=assigned, Set subnet-name, az, date, requestor
      • (add subnet-id after subnet creation as a separate update to prevent deadlocks)
    • 3rd: Create a new block using NetAddr from cidr-vpc-assign table in cidr-subnet-assign table
      • Add the new cidr assignment to cidr-subnet-assign DDB table populating:
        • Status=assigned, region, cidr, vpc-index, account-ou-key, vpc-name, subnet-name, az, date, requester
      • All cidrs with same vpc-index must be unique/no overlap

6. Add new config file variables inside all account objects (accountname\opt-in-vpcs) (Completed in #714 )

• opt-in-vpcs: [“opt-in-vpc-name1”, “opt-in-vpc-name2”, “opt-in-vpc-nameN”]
  • a vpc definition with the specified name, must exist in the ou config for the specified account
  • the vpc definition in the ou must have the opt-in: true set
  • When this parameter exists, create the vpc for the account
  • Allow adding new vpc’s to array, block removing vpc’s from the array unless:
  • { "overrideComparison": true } or {"configOverrides": {'ov-acct-vpc-optin': true } }

7. Near end of state machine, backup all DDB tables to individual files in s3 bucket

8. Enable accounts to request an opt-in VPC - TODO - refine requirements --> DESCOPED FROM CURRENT TASK

• Deploy Service Catalog (SC) item to each account
• SC to offer opt-in VPC options
  • How do we list the options/what is available?
• Sends message to central config file to add new entry in opt-in-vpcs parameter in the specific account
• attains approval, auto-runs
• runs state machine

SAMPLE DDB RECORDS:

DDB table: PBMMAccel-cidr-pools
{
"cidr": "10.0.0.0/13",
"region": "ca-central-1",
"pool": "main"
}
{
"cidr": " 100.96.252.0/23",
"pool": "RFC6598",
"region": "ca-central-1"
}

DDB table: PBMMAccel-cidr-vpc-assign

{
"status": "assigned",
"region": "ca-central-1",
"cidr": "10.1.0.0/16",
"pool": "main",
"index": "548726314785236765",
"account-ou-key": "organizational-unit\Dev",
"vpc-name": "Dev",
"account-id": 111111111111,
"vpc-id": "vpc-096b07371301eeb7f",
"date": "thedatewithtime",
"requester": "Accelerator-Prefix"
}

DDB table: PBMMAccel-cidr-subnet-assign

{
"status": "assigned",
"region": "ca-central-1",
"cidr": "10.1.10.0/24",
"sub-pool": "a",
"vpc-index": "548726314785236765",
"account-ou-key": "organizational-unit\Dev",
"vpc-name": "Dev",
"subnet-name": "Dev",
"az": "a",
"subnet-id": "subnet-00f76121991ce23fa",
"date": "thedatewithtime",
"requester": "Accelerator-Prefix"
}

from aws-secure-environment-accelerator.

Will be in next MAJOR (not minor release) i.e. v1.5 - development complete (excluding item 6 and 8, future dev)

from aws-secure-environment-accelerator.