pulumi / pulumi-awsx Goto Github PK

We are currently not exposing the route tables for a network as properties. This has led to problems for a user who was trying to create VPC Endpoints for S3 on top of a Network, which requires access to the route tables.

The workaround we shared was below - but the middle part of this is complex and should not be required:

let network = new awsinfra.Network(`${prefix}-net`, {
    numberOfAvailabilityZones: config.availabilityZones, // Create subnets in many AZs
    usePrivateSubnets: true,                                // Run compute inside private subnets in each AZ
});

// Define the policy for exposing an S3 VPC endpoint
let endpointPolicy = {
    Version: "2012-10-17",
    Statement: [{
        Action: "*",
        Effect: "Allow",
        Principal: "*",
        Resource: "*",
    }, {
        Action: "*",
        Effect: "Allow",
        Principal: "*",
        Resource: [
            "arn:aws:s3:::*.amazonaws.com",
            "arn:aws:s3:::*.amazonaws.com/*",
        ],
    }],
}

// Lookup the route table for a subnet
async function getRouteTableIdForSubnet(subnetId) {
    let res = await aws.ec2.getRouteTable({
        subnetId: subnetId,
    });
    return res.routeTableId;
}

// Find all route tables for the Network (one for each private subnet, and the shared route table for the public subnets)
let routeTableIds = pulumi.all([...network.subnetIds, network.publicSubnetIds[0]]).apply(subnetIds => {
    return Promise.all(subnetIds.map(getRouteTableIdForSubnet));
});

// Create a VPC Endpoint in the VPC applied to the 
let vpcEndpointS3 = new aws.ec2.VpcEndpoint("s3-endpoint", {
    policy: JSON.stringify(endpointPolicy),
    routeTableIds: routeTableIds,
    vpcId: network.vpcId,
    serviceName: `com.amazonaws.${aws.config.region}.s3`,
});

The tests for this repo are regularly timing out:

...
[ ejs/aws-infra/examples/fargate ] Still running command '/home/travis/.pulumi/bin/pulumi preview --non-interactive --diff' (/tmp/containers-fargate-7e9da657-482048758)...
The job exceeded the maximum time limit for jobs, and has been terminated.

For instance, see https://travis-ci.com/pulumi/pulumi-aws-infra/builds/94905751.

It appears the algorithm for splitting up CIDR blocks within subnets inside of a VPC might be wrong. The following program:

import * as awsx from "@pulumi/awsx";
const vpc = new awsx.ec2.Vpc("my-vpc", {
    cidrBlock: "10.0.0.0/28",
});

yields the following error:

    error: Running program '/Users/joeduffy/temp/awsxplay' failed with an unhandled exception:
    Error: Cidr mask must be between "16" and "28" but was 30
        at VpcTopology.createSubnetsWorker (/Users/joeduffy/temp/awsxplay/node_modules/@pulumi/ec2/vpcTopology.ts:104:19)
        at VpcTopology.createSubnets (/Users/joeduffy/temp/awsxplay/node_modules/@pulumi/ec2/vpcTopology.ts:48:33)
        at new Vpc (/Users/joeduffy/temp/awsxplay/node_modules/@pulumi/ec2/vpc.ts:81:49)
        at Object.<anonymous> (/Users/joeduffy/temp/awsxplay/index.ts:4:13)
        at Module._compile (internal/modules/cjs/loader.js:689:30)
        at Module.m._compile (/Users/joeduffy/temp/awsxplay/node_modules/ts-node/src/index.ts:439:23)
        at Module._extensions..js (internal/modules/cjs/loader.js:700:10)
        at Object.require.extensions.(anonymous function) [as .ts] (/Users/joeduffy/temp/awsxplay/node_modules/ts-node/src/index.ts:442:12)
        at Module.load (internal/modules/cjs/loader.js:599:32)
        at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    error: an unhandled error occurred: Program exited with non-zero exit code: 32

Putting in M23 just for an initial triage given the upcoming launch. I'll try a few other CIDR masks to see whether this happens anytime I venture outside of /16 or whether it's more limited in scope (likely) to choosing an extreme like 28 which is the smallest range allowed.

Part of #51.
Tagging @lukehoban

Would like something briefer than

let overwriteShape: OverwriteShape = undefined!;
let argsShape: ClusterAutoScalingLaunchConfigurationArgs = undefined!;
argsShape = overwriteShape;
overwriteShape = argsShape;

With https://github.com/pulumi/pulumi-awsx/releases/tag/v0.18.3 (Typescript) (and Pulumi CLI 0.17.10), I see some strange diffs to AWS VPC tags when I run pulumi up after a pulumi refresh.

Here is the code that I use to create the VPC:

const vpc = new awsx.ec2.Vpc(`${prefix}-vpc`, {
    // EKS requires at least two availability zones.
    numberOfAvailabilityZones: 3,

    // Ensure that there is a private AND public subnet in each AZ.
    // The Kubernetes worker nodes will be deployed to the private
    // subnet.
    //
    // The load balancers will be created in the public subnet.
    subnets: [{ type: 'public' }, { type: 'private' }],
})

Here is the pulumi up preview right after I run pulumi refresh:

 pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::pulumi:pulumi:Stack::sg-REDACTED-REDACTED-replica-ds-REDACTED-replica-release]
            ~ aws:ec2/route:Route: (update)
                [id=r-rtb-0981bff5ac5188cd91080289494]
                [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-0-nat-0]
              + gatewayId: "nat-07f8b134c241f07f0"
            ~ aws:ec2/route:Route: (update)
                [id=r-rtb-0ffca6caaa8613ba91080289494]
                [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-1-nat-1]
              + gatewayId: "nat-0b09091ded3d19792"
            ~ aws:ec2/route:Route: (update)
                [id=r-rtb-0a449cdf74c14f7601080289494]
                [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-2-nat-2]
              + gatewayId: "nat-06e709f5d14313b1f"
        ~ aws:ec2/securityGroup:SecurityGroup: (update)
            [id=sg-0fba37d58cd28fb7b]
            [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::eks:index:Cluster$aws:ec2/securityGroup:SecurityGroup::sourcegraph-REDACTED-REDACTED-replica-cluster-nodeSecurityGroup]
          ~ tags: {
              - kubernetes                                                                     : {
                  - io/cluster/sourcegraph-REDACTED-REDACTED-replica-cluster-eksCluster-3add07e: "owned"
                }
              + kubernetes.io/cluster/sourcegraph-REDACTED-REDACTED-replica-cluster-eksCluster-3add07e: "owned"
            }

Here is the pulumi up output after that refresh:

  aws:ec2:Route (sourcegraph-REDACTED-REDACTED-replica-vpc-private-0-nat-0):
    error: Plan apply failed: 1 error occurred:
    	* updating urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-0-nat-0: Error: more than 1 target specified. Only 1 of gateway_id, egress_only_gateway_id, nat_gateway_id, instance_id, network_interface_id or vpc_peering_connection_id is allowed.

  aws:ec2:Route (sourcegraph-REDACTED-REDACTED-replica-vpc-private-1-nat-1):
    error: Plan apply failed: 1 error occurred:
    	* updating urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-1-nat-1: Error: more than 1 target specified. Only 1 of gateway_id, egress_only_gateway_id, nat_gateway_id, instance_id, network_interface_id or vpc_peering_connection_id is allowed.

  aws:ec2:Route (sourcegraph-REDACTED-REDACTED-replica-vpc-private-2-nat-2):
    error: Plan apply failed: 1 error occurred:
    	* updating urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-2-nat-2: Error: more than 1 target specified. Only 1 of gateway_id, egress_only_gateway_id, nat_gateway_id, instance_id, network_interface_id or vpc_peering_connection_id is allowed.

  pulumi:pulumi:Stack (sg-REDACTED-REDACTED-replica-ds-REDACTED-replica-release):
    (node:66606) ExperimentalWarning: queueMicrotask() is experimental.

    error: update failed

If I run pulumi up again (right after that last command), this is the pulumi preview output:

Do you want to perform this update? details
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::pulumi:pulumi:Stack::sg-REDACTED-REDACTED-replica-ds-REDACTED-replica-release]
            ~ aws:ec2/route:Route: (update)
                [id=r-rtb-0981bff5ac5188cd91080289494]
                [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-0-nat-0]
              - egressOnlyGatewayId   : ""
              + gatewayId             : "nat-07f8b134c241f07f0"
              - instanceId            : ""
              - natGatewayId          : "nat-07f8b134c241f07f0"
              - networkInterfaceId    : ""
              - transitGatewayId      : ""
              - vpcPeeringConnectionId: ""
            ~ aws:ec2/route:Route: (update)
                [id=r-rtb-0a449cdf74c14f7601080289494]
                [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-2-nat-2]
              - egressOnlyGatewayId   : ""
              + gatewayId             : "nat-06e709f5d14313b1f"
              - instanceId            : ""
              - natGatewayId          : "nat-06e709f5d14313b1f"
              - networkInterfaceId    : ""
              - transitGatewayId      : ""
              - vpcPeeringConnectionId: ""
            ~ aws:ec2/route:Route: (update)
                [id=r-rtb-0ffca6caaa8613ba91080289494]
                [urn=urn:pulumi:ds-REDACTED-replica-release::sg-REDACTED-REDACTED-replica::awsx:x:ec2:Vpc$awsx:x:ec2:Subnet$aws:ec2/route:Route::sourcegraph-REDACTED-REDACTED-replica-vpc-private-1-nat-1]
              - egressOnlyGatewayId   : ""
              + gatewayId             : "nat-0b09091ded3d19792"
              - instanceId            : ""
              - natGatewayId          : "nat-0b09091ded3d19792"
              - networkInterfaceId    : ""
              - transitGatewayId      : ""
              - vpcPeeringConnectionId: ""
           

And that second pulumi update succeeds!

I have not made any changes to the AWS VPC configuration in between updates.

I'm not sure how to reproduce this, but I'm happy to hop on Slack and try to provide any more information.

Unless I'm missing something, it appears that LoadBalancers default external to true, despite the documentation saying that they default to false.

If I create one simply as

new awsx.elasticloadbalancingv2.NetworkLoadBalancer("web-traffic");

and then do an update

new awsx.elasticloadbalancingv2.NetworkLoadBalancer("web-traffic", { external: true });

there is no change to the underlying resource configs.

If I instead then change it to

new awsx.elasticloadbalancingv2.NetworkLoadBalancer("web-traffic", { external: false });

a number of things get changed in the update.

shouldn't really be necessary.

Part of #51.
Tagging @lukehoban

Breaking out of #167

The abstractions for Services and Tasks in @pulumi/cloud are nice, but the focus on being cloud-neutral has led to severe limitations for users who are working in AWS.

We have had many requests for simple, high-level and composable abstractions for working with the building blocks of AWS ECS Services, but without sacrificing the ability to configure the full breadth of AWS capabilities and integrations. This is similar to, and builds on top of, the existing Network and Cluster abstractions, as well as being at the same level of abstraction as the API component in the AWS package.

While motivated by ECS, many of the required building blocks should be usable more generally as high level components that can be composed into other pieces of AWS infrastructure - much like Network is a general purpose building block today.

The areas we want to present simple higher-level APIs include the following (in roughly priority order):

• Network/VPC
• Vpc/Subnet-Placement
• Cluster
• TaskDefinition
• Service
• ECR-based Docker images
• Load Balancing/TargetGroups/Listeners
• SecurityGroups
• AutoScaling Groups
• IAM roles/policies
• Service Discovery
• Metrics
• Alarms
• Dashboard
• RDS/Aurora
• Move things out of 'x' namespace.
• Remove nonsensical instance methods.
• Update examples to not use 'Cloud'.
• Do not use 'instance' as the name of the underlying aws resource. #82

Initial focus is on enabling the same sorts of apps that can be built with cloudaws.Service to be expressed naturally in just a few lines of code using the above abstractions - but without any significant limitations on expressivity relative to AWS capabilities.

As part of this process, we also want to document the API design guidelines for these kinds of APIs to inform future expansions into additional areas of the AWS APIs (and Azure, GCP, Kubernetes and others).

It's quite common to want to add VPC Endpoints to an AWS VPC in order to provide access to AWS services which does not transit the public internet or (from private subnets) the NAT gateway.

Adding an endpoint requires a list of routing tables to modify - generally (but not necessarily) all of the subnets in the VPC - to be passed into a aws.ec2.VpcEndpoint resource. Right now adding gateway endpoints for S3 and DynamoDB can be achieved with the following program:

function addAmazonServiceGateways(namePrefix: string, vpc: awsx.ec2.Vpc) {
    const publicRouteTables = vpc.getSubnets("public").map(x => x.routeTable!.id);
    const privateRouteTables = vpc.getSubnets("private").map(x => x.routeTable!.id);
    const allRouteTables = publicRouteTables.concat(privateRouteTables);

    new aws.ec2.VpcEndpoint(`${namePrefix}-s3-endpoint`, {
        vpcId: vpc.id,
        serviceName: `com.amazonaws.${aws.config.region}.s3`,
        routeTableIds: allRouteTables,
    });

    new aws.ec2.VpcEndpoint(`${namePrefix}-dynamodb-endpoint`, {
        vpcId: vpc.id,
        serviceName: `com.amazonaws.${aws.config.region}.dynamodb`,
        routeTableIds: allRouteTables,
    });
}

It would be nice to add support for this directly to awsx.ec2.Vpc, such that a user can either opt into endpoint creation as part of VpcArgs, or provide method calls to add support for both gateway and interface endpoints and a set of helper functions for constructing the endpoint service names.

From @lukehoban on December 28, 2017 22:36

When setting up a private network, it would be very helpful to have the option to allocate a bastion host with SSH access using a provided key material.

This probably should be part of the Network layer, but could also in theory be part of a high level like the Cluster layer.

Copied from original issue: pulumi/pulumi-cloud#323

Feedback from a community member:

However documentation awsx.ec2.Vpc.fromExistingIds for could be expanded. I am guessing that if I add my existing vpc to my stack, even if i delete my stack the vpc will be left alone? I really don't want to risk deleting our current vpc.
https://pulumi.io/reference/pkg/nodejs/@pulumi/awsx/ec2/#Vpc-fromExistingIds

Without going into the code the behavior of this (and similar) methods is not obvious. We should document these methods, or maybe centrally the naming/usage pattern of methods that create resources versus use existing resources as read-only.

We still have a runtime method run on FargateTaskDefinition which has the same limited capabilities as the version that was in cloud. We should ensure that this functionality is exposed in a rich way aligned with the overall goals of aws-infra.

I see two options:

1. Keep a runtime method, probably naming it runTask for consistency, and have it's arg bag be derived from the full AWS-SDK arg bag with more inputs optional or not settable (in particular launchType and networkConfiguration).
2. Instead, have some way to get the networkConfiguration details off of a cluster so that the underlying runTask can be easily called directly.

The current API has several critical limitations:

1. No way to pass many parameters to run - like command.
2. No way to get the returned TaskARN to use for querying status of the task later.

I found the property templateParameters and corresponding type `TemplateParameters, to be awkwardly named:

I think it's named this way because we happen to implement the auto-scaling group using a CloudFormation template and stack? If yes, shouldn't this be an implementation detail, and we should instead give them more logical names that don't leak this detail?

Target type "lambda" is a valid target type that is missing from targetGroup.d.ts :

    targetType?: pulumi.Input<"instance" | "ip">;

Terraform reference:
https://www.terraform.io/docs/providers/aws/r/lb_target_group.html

EDIT: it seems to only be missing from awsx - you can probably close this

This will be the repo we want for complex large-scale components. 'aws' will be for raw resources, or simple extensions (liek serverless) of raw resources.

Tagging @lukehoban

Opening this to track catching up on READMEs with all of the features added recently. I'm only aware of this - but feel free to add more if there are smaller granularity features which have been added by don't yet have README coverage:

• cloudwatch

Most of our module/namespaces are the shortened abbreviation, like ebs, ecr, ecs, and so on. The one that sticks out here, however, is elasticloadbalancingv2 which is very long. Should we instead call this elbv2 or, since AFAIK v1 is "deprecated" and shouldn't be used (something the more forward-looking AWSX package can take advantage of), just elb?

When creating a VPC with the awsx.ec2.Vpc class, the resulting resources are not receiving the Name tag. Previously the awsx.Network class was creating these tags, which is why I'm curious if this functionality was changed.

Below is the example statement I'm using when creating a VPC:

const vpc = new awsx.ec2.Vpc(
    `secrets-scanner-${env}`,
    {
        cidrBlock: '10.10.0.0/16',
        numberOfAvailabilityZones: 2,
    },
    { protect: true }
);

And here are the versions of the Pulumi libraries I'm using in my package.json:

"dependencies": {
    "@pulumi/aws": "latest",
    "@pulumi/awsx": "^0.16.4",
    "@pulumi/cloud": "^0.16.2",
    "@pulumi/cloud-aws": "^0.16.2",
    "@pulumi/pulumi": "latest"
  }

From @joeduffy on December 4, 2017 23:50

We had a customer request to run C# lambdas from within an otherwise JavaScript Pulumi program. In principle, I see no reason this can't work. In fact, if we do pulumi/pulumi-cloud#168, it seems we would be very close to being able to do so.

The major challenge, I think, is going to be the "shimming" required to present a consistent function signature that is sufficiently insulated from the target cloud environment. But given all the code rewriting we already do, I don't know that this is prohibitively complicated.

Copied from original issue: pulumi/pulumi-cloud#266

The infra.Network class defaults to creating subnets in the form of 10.10.${i}.0/24.

This works fine if using the default VPC CIRD of 10.10.0.0/16, gowever if the VPC is created with a different CIDR block, subnet creation fails.

Example:

const network = new infra.Network("eks-vpc", {
    cidrBlock: "192.168.0.0/16"   
})

Diagnostics:
  aws:ec2:Subnet (eks-vpc-1):
    error: Plan apply failed: Error creating subnet: InvalidSubnet.Range: The CIDR '10.10.1.0/24' is invalid.
    	status code: 400, request id: 9565998e-ad43-4e05-bf7a-e9e02a0c36ee
 
  aws:ec2:Subnet (eks-vpc-0):
    error: Plan apply failed: Error creating subnet: InvalidSubnet.Range: The CIDR '10.10.0.0/24' is invalid.
    	status code: 400, request id: e49b0042-25a0-4eed-a4b2-ebd5ee82f265

It feels surprisingly hard to use new awsinfra.ecs.FargateService with an existing VPC/subnet/SG.

Below is what I ended up with:

const metabaseVpc = awsinfra.ec2.Vpc.fromExistingIds("network", {
    vpcId: "vpc-3567864c",
    publicSubnetIds: ["subnet-53f5b81b", "subnet-469cb120"],
}, { providers: { aws: awsProvider } });

const metabaseSecurityGroup = new awsinfra.ec2.SecurityGroup("metabase", {
    instance: aws.ec2.SecurityGroup.get("metabase", "sg-085f945afdff0ba8b", {}, { provider: awsProvider }),
}, { providers: { aws: awsProvider } });

const metabaseCluster = new awsinfra.ecs.Cluster("metabase", {
    vpc: metabaseVpc,
    securityGroups: [metabaseSecurityGroup],
}, { providers: { aws: awsProvider } });

const metabaseService = new awsinfra.ecs.FargateService("metabase", {
    cluster: metabaseCluster,
    assignPublicIp: true,
    taskDefinitionArgs: {
        containers: {
            metabase: {
                image: "metabase/metabase",
                portMappings: [{ containerPort: 3000 }],
                environment: [{ name: "JAVA_TIMEZONE", value: "US/Pacific" }],
                logConfiguration: {
                    logDriver: "awslogs",
                    options: {
                        "awslogs-group": "/ecs/metabase",
                        "awslogs-region": region,
                        "awslogs-stream-prefix": "ecs",
                    },
                },
            },
        },
    },
}, ({ providers: { aws: awsProvider } } as any));

Several issues:

1. Duplicate URN issues when using existing subnets
2. Wasn't immediately clear why I need to configure a new Vpc object (or Security GRoup) here - I just wanted to pass networkConfiguration directly to new Service, which feels like it woud have been easier with the raw aws.ecs.Service.
3. I'm still getting Error: invocation of aws:ec2/getVpc:getVpc returned an error: transport is closing - not clear if that is due to a bug in the library or something I'm not doing right yet
4. It would be nice to accept strings as input for securityGroups in place of the strongly typed wrappers over security groups - the code above to get an aws.x.ecs.SecurityGroup from a string is pretty awkward.

Part of #51.
Tagging @lukehoban

For the parts of filesystem we currently depend on (the ability to add something to the runcmd block) allow that to be passed in through some interface instead.

The following code snippet seems to cause a 400 error, because the memory for task is not being calculated correctly based on its containers.

  let service = new awsx.ecs.FargateService("sandbox_blah", {
    desiredCount: 2,
    taskDefinitionArgs: {
      containers: {
        blah: {
          image: "nginx",
          memory: 1024,
          portMappings: [blahTLSListener],

Error message for fargate cluster task
  aws:ecs:TaskDefinition (sandbox_blah):
    error: Plan apply failed: ClientException: The 'memory' setting for container 'blah' is greater than for the task.
    	status code: 400, request id: 5b315435-4e8f-11e9-902b-637d0e80e28d

Adding

    taskDefinitionArgs: {
      memory: "2048",

fixes the issue.

It would be nice if numberOfAvailabilityZones automatically used the number of AZs in my current region, or at least offered the option to do so.

Unfortunately, because this is a number, doing so manually requires a bit of async tedium:

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

module.exports = (async () => {
    const azs = await aws.getAvailabilityZones();
    const vpc = new awsx.ec2.Vpc("my-vpc", {
        numberOfAvailabilityZones: azs.names.length,
    });
    return {
        vpcId: vpc.id,
    };
})();

This is certainly not impossible, but any of the following would have been much nicer:

_Option 1. Automatically use all AZs by default.

import * as awsx from "@pulumi/awsx";
const vpc = new awsx.ec2.Vpc("my-vpc");
export vpcId = vpc.id;

_Option 2. Accept an auto parameter for numberOfAvailabilityZones (opt in).

import * as awsx from "@pulumi/awsx";
const vpc = new awsx.ec2.Vpc("my-vpc", { numberOfAvailabilityZones: "auto" });
export vpcId = vpc.id;

_Option 3. Accept a Promise<number> for numberOfAvailabilityZones.

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";
const vpc = new awsx.ec2.Vpc("my-vpc", {
    numberOfAvailabilityZones: aws.getAvailabilityZones().then(az => az.names.length),
});
export vpcId = vpc.id;

We've adopted a pattern of using the name instance to refer to the raw resource that is morally the key thing wrapped by a higher level component in this library - like awsinfra.ecs.Cluster#instance will return the aws.ecs.Cluster.

This was done to avoid stuttering like cluster.cluster in common cases.

However, I would suggest that it is better to go with the cluster.cluster approach for a variety of reasons. It's not perfect, but I believe on balance it's the best option.

1. Typically this is just one of several underlying resources being managed, and although its the “main” one, it’s awkward to name it differently. Everything else is securityGroup, or subnet, but then the cluster is instance.
2. We've consistently seen everyone initially go with the cluster.cluster style naming by default, since it is the most natural option - and I think anything else is likely to not be adopted broadly enough.
3. The repetition just isn't that bad, and actually has a level of simple clarity to it - it's likely to be more discoverable and usable than alternatives.

On an external ALB, the following code, does not modify the ALB security group rules, but the docs suggest it should.

  const myListener = myTargetGroup.createListener(
    "my_redirect",
    {
      port: 80,
      external: true,
      protocol: "HTTP",
      defaultAction: {
        type: "redirect",
        redirect: {
          port: "443",
          protocol: "HTTPS",
          statusCode: "HTTP_301"
        }
      }
    }
  );

package.json

{
  "name": "typescript",
  "devDependencies": {
    "@types/node": "10.14.3"
  },
  "dependencies": {
    "@pulumi/aws": "0.17.1",
    "@pulumi/awsx": "0.17.3",
    "@pulumi/pulumi": "0.17.2"
  }
}

NOTE: Still happens on 0.18 releases of aws/awsx, just tried them out just in case.

It is common to configure a DHCP Options Set when creating a VPC, which allows DNS Servers, a custom search domain, NTP servers and a few other parameters to instances via the DHCP system built into the VPC control plane.

Currently the following function can be used to add an options set for a custom search domain.

function addDhcpOptions(namePrefix: string, baseTags: aws.Tags, zoneName: string, vpc: awsx.ec2.Vpc): pulumi.Output<string> {
    const privateZone = new aws.route53.Zone(`${namePrefix}-private-hosted-zone`, {
        vpcs: [{
            vpcId: vpc.id,
        }],
        name: zoneName,
        comment: `Private zone for ${zoneName}. Managed by Pulumi.`,
    }, {parent: vpc});

    const dhcpOptionSet = new aws.ec2.VpcDhcpOptions(`${namePrefix}-dhcp-options`, {
        domainName: privateZone.name,
        domainNameServers: ["AmazonProvidedDNS"],
        tags: Object.assign({
            Name: `DHCP Options (${zoneName})`,
        }, baseTags),
    }, vpc);

    new aws.ec2.VpcDhcpOptionsAssociation(`${namePrefix}-dhcp-options-assoc`, {
        vpcId: vpc.id,
        dhcpOptionsId: dhcpOptionSet.id,
    }, {parent: dhcpOptionSet});

    return privateZone.zoneId;
}

Note: the above program also creates the zone itself, which I do not believe should be done by the Vpc class.

It would be nice to allow a zone ID or name to be passed to the VpcArgs of awsx.ec2.Vpc in order that this can be done automatically, and that the zone ID can be exposed as one of the outputs of the Vpc instead of being passed around separately to create records on it.

Part of #51.
Tagging @lukehoban

For the parts of cluster we currently dpeend on (only security groups i believe), allow that to be passed in through some interface instead.

Part of #51.
Tagging @lukehoban

Given something like this:

const metabaseVpc = awsinfra.ec2.Vpc.fromExistingIds("network", {
    vpcId: "vpc-3567864c",
    publicSubnetIds: ["subnet-53f5b81b", "subnet-469cb120"],
}, { providers: { aws: awsProvider } });

Will result in duplicate URN errors error: Duplicate resource URN 'urn:pulumi:datalake-testing::datalake::awsinfra:x:ec2:Vpc$awsinfra:x:ec2:Subnet::public-1'; try giving it a unique name

This is because two resources with the same name are created in createSubnets:

https://github.com/pulumi/pulumi-aws-infra/blob/17bf2b18e5671819b9b65e984cbe1e89a8e60475/nodejs/aws-infra/experimental/ec2/vpc.ts#L245-L247

Following docs at https://github.com/pulumi/pulumi-awsx/tree/master/nodejs/awsx/ecs#clusters:

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

const vpc = // ... create custom vpc
const cluster = new awsx.ecs.Cluster("custom", { vpc });

const asg = cluster.createAutoScalingGroup("custom", {
    templateParameters: { minSize: 20 },
    launchConfigurationArgs: { instanceType: "t2.medium" },
});

Leads to:

error: Plan apply failed: 1 error occurred:
        * creating urn:pulumi:fib-dev-ecs::fib::awsx:x:ecs:Cluster$awsx:x:autoscaling:AutoScalingGroup$aws:cloudformation/stack:Stack::app-asg: ROLLBACK_COMPLETE: ["The following resource(s) failed to create: [Instances]. . Rollback requested by user." "Property AvailabilityZones cannot be empty."]

And indeed, the outputted CloudFormation template is missing AvailabilityZones:

    AWSTemplateFormatVersion: '2010-09-09'
    Outputs:
        Instances:
            Value: !Ref Instances
    Resources:
        Instances:
            Type: AWS::AutoScaling::AutoScalingGroup
            Properties:
                Cooldown: 300
                DesiredCapacity: 2
                HealthCheckGracePeriod: 120
                HealthCheckType: EC2
                LaunchConfigurationName: "app-asg-1ff1dbd"
                MaxSize: 100
                MetricsCollection:
                -   Granularity: 1Minute
                MinSize: 2
                VPCZoneIdentifier: []
                Tags:
                -   Key: Name
                    Value: app-asg
                    PropagateAtLaunch: true
            CreationPolicy:
                ResourceSignal:
                    Count: 2
                    Timeout: PT15M
            UpdatePolicy:
                AutoScalingRollingUpdate:
                    MaxBatchSize: 1
                    MinInstancesInService: 2
                    PauseTime: PT15M
                    SuspendProcesses:
                    -   ScheduledActions
                    WaitOnResourceSignals: true

From @lukehoban on November 12, 2017 18:49

Our Cluster primitive doesn't currently autoscale. It's uses an ASG just as a way to template out VMs, but doesn't have any dynamic scaling policy applied.

We should presumably set some scaling policy (and potentially allow high-level parameterization of that). At the very least, scaling on memory reservation seems natural, as we will hard fail to launch new containers if we don't scale with load on that metric.

Copied from original issue: pulumi/pulumi-cloud#209

The build for this repo appears to be broken:

experimental/elasticloadbalancingv2/targetGroup.ts:60:29 - error TS2532: Object is possibly 'undefined'.
60             containerPort: +port,
                               ~~~~
experimental/elasticloadbalancingv2/targetGroup.ts:65:9 - error TS2322: Type 'Output<{ containerPort: Output<number | undefined>; targetGroupArn: Output<string>; }[]>' is not assignable to type 'Input<Input<ContainerLoadBalancer>[]>'.
  Type 'Output<{ containerPort: Output<number | undefined>; targetGroupArn: Output<string>; }[]>' is not assignable to type 'Output<Input<ContainerLoadBalancer>[]>'.
    Type '{ containerPort: Output<number | undefined>; targetGroupArn: Output<string>; }[]' is not assignable to type 'Input<ContainerLoadBalancer>[]'.
      Type '{ containerPort: Output<number | undefined>; targetGroupArn: Output<string>; }' is not assignable to type 'Input<ContainerLoadBalancer>'.
        Type '{ containerPort: Output<number | undefined>; targetGroupArn: Output<string>; }' is not assignable to type 'ContainerLoadBalancer'.
          Types of property 'containerPort' are incompatible.
            Type 'Output<number | undefined>' is not assignable to type 'Input<number>'.
              Type 'Output<number | undefined>' is not assignable to type 'Output<number>'.
                Type 'number | undefined' is not assignable to type 'number'.
                  Type 'undefined' is not assignable to type 'number'.
65         return this.dependencies().apply(_ => [{
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
66             containerPort: this.instance.port,
   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
67             targetGroupArn: this.instance.arn,
   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
68         }]);
   ~~~~~~~~~~~~
Found 2 errors.

The Vpc object makes working with newly created VPCs easy -- you get to specify the number of public/private/isolated subnets and automatically have NAT and/or Internet gateways automatically created. My use case requires us to work within an existing VPC so we do not have access to all those niceties provided by the Vpc object -- they are only available when creating a new Vpc.

My suggestion is to expose the subnet, NAT gateway, and Internet gateway creation as methods on the Vpc object. For example:

const vpc = infra.ec2.Vpc.fromExistingIds("my-vpc", {
    vpcId: myExistingVpcId,
    publicSubnetIds: mySubnets,
});

vpc.addSubnet()
vpc.subnets[0].withNatGateway()
vpc.addNatGatewayToPrivateSubnets
vpc.addInternetGateway

Some additional areas we would like to add additional high level components:

• IAM roles/policies
• Service Discovery
• Metrics. #214
• Alarms. #214
• Scaling policies. #240 and #232
• Dashboard. #217
• RDS/Aurora

API keys are one of a few features of API Gateway that are not exposed yet in the higher-level API component. We should expose a way to use these (and other features), ideally in such a way that we do not need to duplicate much/any functionality that is already available from API gateway.

Hello,

I am trying to create an EKS cluster mostly following this example: https://github.com/pulumi/examples/tree/master/aws-ts-eks.

However, I want to create a cluster with a VPC that has both a public and private subnet in each AZ (following the best practices as described in the "Networking" section of https://blog.aquasec.com/first-steps-creating-and-securing-an-eks-cluster).

I am using the awsx.ec2.VPC component provided by this package in order to create that kind of VPC since the example in the Readme looks like exactly what I need: https://github.com/pulumi/pulumi-awsx/blame/a4b32654466961f77de68c4f5ed90a0cbc747514/nodejs/awsx/ec2/README.md#L66

I've created a repository that has the exact configuration that I'm trying to use: https://github.com/ggilmore/pulumi-eip-repro

I keep getting the following error whenever I run pulumi up in https://github.com/ggilmore/pulumi-eip-repro:

Diagnostics:
  aws:ec2:Eip (eksNetwork-0):

    error: Plan apply failed: 1 error occurred:
    * creating urn:pulumi:dev::eks-cluster::awsx:x:ec2:NatGateway$aws:ec2/eip:Eip::eksNetwork-0: Error creating EIP tags: InvalidID: The ID '54.245.84.217' is not valid
        status code: 400, request id: 1fc802a4-c586-4700-9ee2-553d53a4ecd6
  aws:ec2:Eip (eksNetwork-1):

    error: Plan apply failed: 1 error occurred:
    * creating urn:pulumi:dev::eks-cluster::awsx:x:ec2:NatGateway$aws:ec2/eip:Eip::eksNetwork-1: Error creating EIP tags: InvalidID: The ID '50.112.109.185' is not valid
        status code: 400, request id: 7dc40f73-2a75-4e12-87de-8fddfa48b406

Full pulumi up output:

Previewing update (dev):
 Type                                    Name                             Plan       Info


pulumi:pulumi:Stack                     eks-cluster-dev                  create     Method handler diffConfig for /pulumirpc.ResourceProvider/
├─ eks:index:Cluster                    cluster                          create
│  ├─ eks:index:ServiceRole             cluster-eksRole                  create
│  │  ├─ aws:iam:Role                   cluster-eksRole-role             create
│  │  ├─ aws:iam:RolePolicyAttachment   cluster-eksRole-4b490823         create
│  │  └─ aws:iam:RolePolicyAttachment         cluster-eksRole-90eb1c99         create
│  ├─ eks:index:ServiceRole                   cluster-instanceRole             create
│  │  ├─ aws:iam:Role                         cluster-instanceRole-role        create
│  │  ├─ aws:iam:RolePolicyAttachment         cluster-instanceRole-3eb088f2    create
│  │  ├─ aws:iam:RolePolicyAttachment         cluster-instanceRole-03516f97    create
│  │  └─ aws:iam:RolePolicyAttachment         cluster-instanceRole-e1b295bd    create
│  ├─ kubernetes:yaml:ConfigGroup             cluster-dashboard                create
│  ├─ pulumi-nodejs:dynamic:Resource          cluster-cfnStackName             create
│  ├─ pulumi-nodejs:dynamic:Resource          cluster-cfnStackName             create
│  ├─ aws:ec2:SecurityGroup                   cluster-eksClusterSecurityGroup  create
│  ├─ aws:ec2:SecurityGroup                                cluster-eksClusterSecurityGroup           create
│  ├─ aws:ec2:SecurityGroup                                cluster-eksClusterSecurityGroup           create
│  ├─ aws:eks:Cluster                                          cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                          cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                             cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                             cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                             cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                             cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                             cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                             cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                             cluster-eksCluster                        create
│  ├─ aws:eks:Cluster                                             cluster-eksCluster                        create
pulumi:pulumi:Stack                                               eks-cluster-dev                           create     2 messages
│  ├─ pulumi:providers:kubernetes                                 cluster-eks-k8s                           create
│  ├─ pulumi-nodejs:dynamic:Resource                              cluster-vpc-cni                           create
│  ├─ aws:ec2:SecurityGroup                                       cluster-nodeSecurityGroup                 create
│  ├─ kubernetes:storage.k8s.io:StorageClass                      cluster-gp2                               create
│  ├─ kubernetes:core:ConfigMap                                   cluster-nodeAccess                        create
│  ├─ aws:ec2:SecurityGroupRule                                   cluster-eksClusterIngressRule             create
│  ├─ aws:ec2:LaunchConfiguration                                 cluster-nodeLaunchConfiguration           create
│  ├─ aws:cloudformation:Stack                                    cluster-nodes                             create
│  ├─ pulumi:providers:kubernetes                                 cluster-provider                          create
│  ├─ kubernetes:core:ServiceAccount                              cluster-eks-admin                         create
│  └─ kubernetes:rbac.authorization.k8s.io:ClusterRoleBinding     cluster-eks-admin                         create
├─ awsx:x:ec2:Vpc                                                 eksNetwork                                create
│  ├─ awsx:x:ec2:Subnet                                           eksNetwork-public-0                       create
│  │  ├─ aws:ec2:RouteTable                                       eksNetwork-public-0                       create
│  │  ├─ aws:ec2:Subnet                                           eksNetwork-public-0                       create
│  │  └─ aws:ec2:RouteTableAssociation                            eksNetwork-public-0                       create
│  ├─ awsx:x:ec2:Subnet                                           eksNetwork-public-1                       create
│  │  ├─ aws:ec2:RouteTable                                       eksNetwork-public-1                       create
│  │  ├─ aws:ec2:Subnet                                           eksNetwork-public-1                       create
│  │  └─ aws:ec2:RouteTableAssociation                            eksNetwork-public-1                       create
│  ├─ awsx:x:ec2:Subnet                                           eksNetwork-private-0                      create
│  │  ├─ aws:ec2:RouteTable                                       eksNetwork-private-0                      create
│  │  ├─ aws:ec2:Subnet                                           eksNetwork-private-0                      create
│  │  └─ aws:ec2:RouteTableAssociation                            eksNetwork-private-0                      create
│  ├─ awsx:x:ec2:Subnet                                           eksNetwork-private-1                      create
│  │  ├─ aws:ec2:RouteTable                                       eksNetwork-private-1                      create
│  │  ├─ aws:ec2:Subnet                                           eksNetwork-private-1                      create
│  │  └─ aws:ec2:RouteTableAssociation                            eksNetwork-private-1                      create
│  └─ awsx:x:ec2:InternetGateway                                  eksNetwork                                create
│     └─ aws:ec2:InternetGateway                                  eksNetwork                                create
├─ awsx:x:ec2:NatGateway                                          eksNetwork-1                              create
│  ├─ aws:ec2:Eip                                                 eksNetwork-1                              create
│  └─ aws:ec2:NatGateway                                          eksNetwork-1                              create
├─ awsx:x:ec2:NatGateway                                          eksNetwork-0                              create
│  ├─ aws:ec2:Eip                                                 eksNetwork-0                              create
│  └─ aws:ec2:NatGateway                                          eksNetwork-0                              create
├─ aws:ec2:Vpc                                                    eksNetwork                                create
├─ aws:ec2:Route                                                  eksNetwork-public-0-ig                    create
├─ aws:ec2:Route                                                  eksNetwork-public-1-ig                    create
├─ aws:ec2:Route                                                  eksNetwork-private-0-nat-0                create
└─ aws:ec2:Route                                                  eksNetwork-private-1-nat-1                create

Diagnostics:

pulumi:pulumi:Stack (eks-cluster-dev):

Method handler checkConfig for /pulumirpc.ResourceProvider/CheckConfig expected but not provided

Method handler diffConfig for /pulumirpc.ResourceProvider/DiffConfig expected but not provided
Resources:

+ 69 to create
Do you want to perform this update? yes

Updating (dev):
 Type                              Name                  Status                  Info


pulumi:pulumi:Stack               eks-cluster-dev       created                 2 messages
├─ awsx:x:ec2:NatGateway          eksNetwork-0          created
│  └─ aws:ec2:Eip                 eksNetwork-0          creating failed     1 error
├─ awsx:x:ec2:NatGateway          eksNetwork-1          created
│  └─ aws:ec2:Eip                 eksNetwork-1          creating failed     1 error
├─ awsx:x:ec2:Vpc                 eksNetwork            created
│  ├─ awsx:x:ec2:Subnet           eksNetwork-public-0   created
│  ├─ awsx:x:ec2:Subnet           eksNetwork-private-1  created
│  ├─ awsx:x:ec2:Subnet           eksNetwork-public-1   created
│  ├─ awsx:x:ec2:InternetGateway  eksNetwork            created
│  └─ awsx:x:ec2:Subnet           eksNetwork-private-0  created
├─ eks:index:Cluster              cluster               created
└─ aws:ec2:Vpc                    eksNetwork            created

Diagnostics:

aws:ec2:Eip (eksNetwork-0):

error: Plan apply failed: 1 error occurred:
* creating urn:pulumi:dev::eks-cluster::awsx:x:ec2:NatGateway$aws:ec2/eip:Eip::eksNetwork-0: Error creating EIP tags: InvalidID: The ID '54.245.84.217' is not valid
	status code: 400, request id: 1fc802a4-c586-4700-9ee2-553d53a4ecd6

aws:ec2:Eip (eksNetwork-1):

error: Plan apply failed: 1 error occurred:
* creating urn:pulumi:dev::eks-cluster::awsx:x:ec2:NatGateway$aws:ec2/eip:Eip::eksNetwork-1: Error creating EIP tags: InvalidID: The ID '50.112.109.185' is not valid
	status code: 400, request id: 7dc40f73-2a75-4e12-87de-8fddfa48b406

pulumi:pulumi:Stack (eks-cluster-dev):

Method handler checkConfig for /pulumirpc.ResourceProvider/CheckConfig expected but not provided

Method handler diffConfig for /pulumirpc.ResourceProvider/DiffConfig expected but not provided
Resources:

+ 11 created
Duration: 7s
Permalink: https://app.pulumi.com/ggilmore/eks-cluster/dev/updates/3

error: update failed

Can someone tell me if there is something wrong with the way that I declared the VPC resource in https://github.com/ggilmore/pulumi-eip-repro ?

Part of #51.
Tagging @lukehoban

Breaking out of #167

This applies to IImageProvider and ILoadBalancerProvider

Part of #51.
Tagging @lukehoban

Today, we restrict numberOfAvailabilityZones to be between 1 and 4. In practice though, this limit is dependent on the target region. We could look this up and use that as the limit instead to get a better early error (the user will still get an error eventually if they exceed the number of AZs in the region).

Today we have awsx.ecs.Image which provides some convenience APIs for managing Docker images. However, it primarily is an ECR feature not an ECS feature, and may make sense to offer a more generalized ECR-focused API over this to make it simple to provision and push to ECR registries. This is particularly important for use with EKS and other Kubernetes deployments on AWS.

i.e. rather than just specifying a count, allow the user to specify [ "us-east-1b", "us-east-1c" ] or similar.

Presumably part of the ECS layer. Breaking this out of #167

For tuesday design meeting.

Ideas with examples. When creating a component that enhances an AWS resource, subclassing provides a mechanism to augment and improve on the resource. This is the approach we took for CallbackFunction (which is-a LambdaFunction). For example:

export declare abstract class ClusterTaskDefinition extends aws.ecs.TaskDefinition {
    readonly cluster: module.Cluster;
    readonly logGroup: aws.cloudwatch.LogGroup;
    readonly containers: Record<string, module.ContainerDefinition>;
    readonly taskRole: aws.iam.Role;
    readonly executionRole: aws.iam.Role;

Benefits:

1. Instances follow an is-a relationship. Code that needs to take an aws.ecs.Cluster can take instances of the subtype.
2. No extra pulumi resource needed to wrap raw AWS resource. Ancillary benefit of this is no need to do things like have code that does cluster.cluster.
3. All expected Output properties can be easily accessed off the instance.
4. Rich auxiliary resources can be easily exposed.

Cons:

1. parenting relationship can be confusing. For example, when creating a rich component that needs to create other resources up front, parenting of these resources is not associated with the expected resource.

--

Construction of one of these subtypes happens by feeding in an args type that is created with the following pattern:

export declare type ClusterTaskDefinitionArgs = utils.Overwrite<aws.ecs.TaskDefinitionArgs, {
    ...

Just as the component type subclasses the AWS type, the argument type is related to the AWS argument type. We use 'Overwrite' to both add new properties, change existing properties, or remove properties altogether. For example:

export declare type ClusterTaskDefinitionArgs = utils.Overwrite<aws.ecs.TaskDefinitionArgs, {
    /** Not used.  The pulumi resource name will be used for this. */
    family?: never;
    /** Not used.  Provide  [containers] instead. */
    containerDefinitions?: never;

'never' properties are how we remove properties that should not longer be provided. These are often for properties we can provide entirely ourselves (like passing launchType: "FARGATE" for FargateServices), or where we can provide a better a better shaped programming model. For example, containerDefinitions is an Input<string>. we want to give a strong type there.

Adding new properties can be done simply like so:

export declare type ClusterTaskDefinitionArgs = utils.Overwrite<aws.ecs.TaskDefinitionArgs, {
...
    /**
     * All the containers to make a ClusterTaskDefinition from.  Useful when creating a
     * ClusterService that will contain many containers within.
     */
    containers: Record<string, module.ContainerDefinition>;

Properties should be changed to be optional if we can provide a sensible default. For example:

export type ClusterAutoScalingLaunchConfigurationArgs = Overwrite<aws.ec2.LaunchConfigurationArgs, {
    ...
    /** The size of instance to launch.  Defaults to t2.micro if unspecified. */
    instanceType?: pulumi.Input<aws.ec2.InstanceType>;

In LaunchConfigurationArgs, 'instanceType' is required. But we can supply something sensible, so we make the property optional. How this is handled in code is shown in the next post.

--

When possible, if the implementation needs a resource that it can create, it should also take an optional input parameters to allow the caller to pass it in. For example:

export declare type ClusterTaskDefinitionArgs = utils.Overwrite<aws.ecs.TaskDefinitionArgs, {
...
    /**
     * Log group for logging information related to the service.  If not provided a default instance
     * with a one-day retention policy will be created.for no log group.
     */
    logGroup?: aws.cloudwatch.LogGroup;

If an implementation may end up creating resources, it should expose those child resources on itself. Using the 'logGroup' example above, this resource would be exposed as:

export declare abstract class ClusterTaskDefinition extends aws.ecs.TaskDefinition {
    ...
    readonly logGroup: aws.cloudwatch.LogGroup;

This ensures clients can always get at resources created under the covers.

--

When there are a set of resource properties that need to be provided/kept-in-sync, this is a good candidate for additional subclasses. For example, users can make fargate task-definitions and ec2 task-definitions. Each of these places additional constraints on properties. This can be exposed like so (using the above rules):

export declare type FargateTaskDefinitionArgs = utils.Overwrite<module.ClusterTaskDefinitionArgs, {
    /** Not provided.  Defaults automatically to ["FARGATE"] */
    requiresCompatibilities?: never;
    /** Not provided.  Defaults automatically to "awsvpc" */
    networkMode?: never;
...
}>;

export declare class FargateTaskDefinition extends module.ClusterTaskDefinition {
    ...
}

When appropriate, these resources should expose relevant inside/outside instance APIs for consumers. For example:

export declare class FargateTaskDefinition extends module.ClusterTaskDefinition {
...
     /** Deployment time api. Creates a service with this as its task definition.  */
    createService(name: string, args: module.FargateServiceArgs, opts?: pulumi.ResourceOptions): module.FargateService;

    /** Runtime api. Runs this task definition in this cluster once. */
    run(options?: TaskRunOptions): Promise<void>;
}

These instance methods both allow for powerful functionality, but they also help one easily compose powerful constructs in a simple manner. For example, using this pattern in more locations, one can write:

    const service = awsinfra.Network.getDefault()
                                    .createCluster("mycluster2", { ... })
                                    .createFargateService("examples-nginx2", { ...

When creating APIs that need to consume other APIs, providing these instance methods helps with discovery. Alternatively once can write:

    const network = awsinfra.Network.getDefault();
    const cluster = new x.Cluster("mycluster", { network: network, ... });
    const service = new x.FargateService("nginx", { cluster: cluster, ... });

This requires more explicit passing of data, but works well as well.

According to the AWS docs, you should be able to tag services and task definitions. The awsx ECS resources do not have support for those right now. I believe this would require adding the tags property to the following interfaces:

• FargateService
• FargateTaskDefinitionArgs

There is a wealth of bucket options that are necessary to set for PWA purposes. For example:

    readonly website?: pulumi.Input<{
        errorDocument?: pulumi.Input<string>;
        indexDocument?: pulumi.Input<string>;
        redirectAllRequestsTo?: pulumi.Input<string>;
        routingRules?: pulumi.Input<string>;
    }>;

Right now we hide the bucket creation and don't give the user any facility to tweak these values. We should make these options optionally available to the user when they create the apigateway.