Giter VIP home page Giter VIP logo

iann0036 / awsconsolerecorder Goto Github PK

View Code? Open in Web Editor NEW
1.4K 43.0 87.0 9.69 MB

Records actions made in the AWS Management Console and outputs the equivalent CLI/SDK commands and CloudFormation/Terraform templates.

License: MIT License

JavaScript 29.14% CSS 62.47% HTML 0.19% Shell 0.01% Less 3.96% SCSS 4.22%
aws cloudformation aws-sdk aws-console aws-cli boto3 chrome-extension firefox-addon terraform troposphere

awsconsolerecorder's Introduction

WebsiteTwitterLinkedInContact

About

Hi there 👋

I'm Ian, an AWS Community Hero, AWS Ambassador and general cloud enthusiast. I work in Sydney as the Cloud Principal at Kablamo, a cloud consultancy with a focus on delivering digital products and solutions, but I also regularly contribute to open-source projects here with a focus on AWS automation and tooling, especially involving cloud security and Infrastructure-as-Code.

I also enjoy speaking at meetups, co-hosting podcasts, engaging with the community on Slack, and posting about my latest experiences within AWS on my blog. You can find more of my work below.

My Work

awsconsolerecorder's People

Contributors

iann0036 avatar lyletagawa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

awsconsolerecorder's Issues

AWS config Rules

Describe the bug
A clear and concise description of what the bug is.

Related Mapping
AWS Config Rules
Related Language
Terraform
Steps to reproduce the behavior:

  1. Go to 'aws console'
  2. create AWS config.
  3. Add a new rule like s3 XX .
  4. Terraform output not getting generated.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Cloudformation for service catalog product - error in array names

Describe the bug
When uploading a new product in service catalog, following cloudformation snippet is created:

    Type: "AWS::ServiceCatalog::CloudFormationProduct"
    Properties:
      Name: "EC2-instance-v1"
      Description: ""
      Owner: "owner1"
      Distributor: ""
      SupportEmail: "[email protected]"
      ProvisioningArtifactParameters: 
          type: "CLOUD_FORMATION_TEMPLATE"
          name: "EC2-instance-v1"
          description: ""
          info: 
            LoadTemplateFromURL: "https://s3.amazonaws.com/cf-templates-1wfdfsyejhajha-eu-west-1/servicecatalog-product-20fdsfsdfsd_catalog.yml"

Related Mapping
AWS::ServiceCatalog::CloudFormationProduct

Related Language
cloudformation

To Reproduce
Steps to reproduce the behavior:

  1. Go to service catalog
  2. Click on upload product
  3. See error in console recorder for cloudformation

Expected behavior
Output for ProvisioningArtifactParameters

     Type: "AWS::ServiceCatalog::CloudFormationProduct"
    Properties:
      Name: "EC2-instance-v1"
      Description: ""
      Owner: "owner1"
      Distributor: ""
      SupportEmail: "[email protected]"
      ProvisioningArtifactParameters: 
          - 
            Name: "EC2-instance-v1"
            Description: ""
            Info: 
              LoadTemplateFromURL: "https://s3.amazonaws.com/cf-templates-1wfdfsyejhajha-eu-west-1/servicecatalog-product-20fdsfsdfsd_catalog.yml"
  1. type seems not supported
  2. capital letters needed for Name,Description,Info.....
  3. "-" to define array

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

CodeDeploy is using the wrong aws cli command codedeploy instead of deploy

Describe the bug
CodeDeploy is using the wrong aws cli command codedeploy instead of deploy

Related Mapping
CodeDeploy

Related Language
AWS CLI
https://docs.aws.amazon.com/cli/latest/reference/deploy/index.html

To Reproduce
Steps to reproduce the behavior:

  1. To the aws codedeploy and list deployments
  2. In AWS Cli recording tab: aws codedeploy list-deployments --region eu-west-1

Expected behavior
aws deploy list-deployments --region eu-west-1

Cognito User Pool: App Integration and Federation actions not captured

Describe the bug
Several of the screens in the Cognito User Pool console (post-creation) are not captured.

To Reproduce
Steps to reproduce the behavior:

  1. Go to 'Cognito User Pools' in the console.
  2. Click on an existing User Pool with an App Client.
  3. Scroll down to 'Federation' and fill out the two pages, e.g. for Google.
  4. Scroll up to 'App Integration' and fill out the App Client Settings, Domain Name and UI Customization
  5. See none of steps 3 and 4 captured.

Create Glue Crawler is missed

Describe the bug
When creating a Glue Crawler from the console, the call to create the Crawler itself is missed.

Related Mapping
glue.CreateCrawler

Related Language
n/a

To Reproduce

  1. Go to https://console.aws.amazon.com/glue/home?region=us-east-1#catalog:tab=crawlers
  2. Click Add Crawler
  3. Fill out required information
  4. Click Add Crawler

Expected behavior
Glue Crawler would be created in the resulting code. Currently the secondary items are created successfully (Glue Connection, Glue Database) when using the same wizard, but not the Crawler itself.

Screenshots
n/a

Additional context
n/a

Replicate Edits

The ability to edit the parameters or content of a mapping type and have that replicate to the other mappings.

Relates to ability to translate between mappings as a seperate project / fork.

Ability to filter non-mutable requests

Many read-only requests generated from navigating the console that people might want to filter out. This reduces output and makes it easier to identify which modifying requests are made.

Empty Popup when pinned to Overflow Menu in Firefox

When you pin the extension to the Overflow Window in Firefox 72 (right-click > Pin to Overflow Menu) only a white empty popup is shown. Works fine when unpinned.

Screenshot 2020-01-31 at 11 49 11

(ACR 0.3.31 in Firefox 72.0.2 on OS X 10.14.6)

Multiple Lambda functions creation

Describe the bug
When I created a Lambda function, I see 5 functions created instead.

Related Mapping
lambda.CreateFunction

Related Language
All of them

To Reproduce
Steps to reproduce the behavior:

  1. Go to https://console.aws.amazon.com/lambda/
  2. Click on 'Create function' (top right)
  3. Configure a Function a name and runtime
  4. Click on 'Create function' (bottom right)

Expected behavior
The function is created correctly, but in the recorded calls I see 5 function creations.

Additional context
You can find the recording here.

UserPool and UserPoolUser resources not picked up

Describe the bug
When creating a userpool using the AWS cognito console, or adding a user to an existing userpool, (AWS::Cognito::UserPool and AWS::Cognito::UserPoolUser), the Recorder plugin indicates "no recorded actions yet" in the cloudformation when recording is stopped.

Related Language
CloudFormation

To Reproduce
Steps to reproduce the behavior:

  1. Start recording on the plugin
  2. Go to the AWS cognito console
  3. Click on "Manage Userpools"
  4. Click on "Create a user pool"
  5. Enter a Pool name and follow either the "Review defaults" or "Step through settings" routes to create a user pool.
    (this creates a UserPool resource, which is not reflected when the console recorder is stopped)
  6. After the pool was created, click on "Users and groups"
  7. Click on "Create user"
  8. Complete the Create user form and click on "Create user".
    (this creates a UserPoolUser resource, which is not reflected when the console recorder is stopped)

Expected behavior
When the console recorder recording is stopped, the dashboard is expected to show a CloudFormation script of the UserPool and UserPoolUser created.

Additional context
This was test on both Chrome and Firefox.

CFN AWS::CodePipeline::Pipeline mapping invalid

Describe the bug
Whilst creating CodePipelines I've found that the CloudFormation output doesn't map the Stage properties correctly. The parameters within the Stages field are camelcase and not uppercase as per the YAML standard.

Related Mapping
AWS::CodePipeline::Pipeline

  • Properties: Stages.Name, Stages.Actions.*

Related Language
CFN yml

To Reproduce
Steps to reproduce the behavior:
Create a CodePipeline, check CF YML Stages.

Screenshots
screen shot 2019-02-26 at 4 05 24 pm

Go SDK outputs are mismatched

Describe the bug
Many Go object parameters are incorrect.

Related Mapping
All

Related Language
Go

To Reproduce
Example:

        Owner: []*string{
            aws.String("self"),
        },
        Filter: []*ec2.Filter{
            &ec2.Filter{
                Name: aws.String("ImageState"),
                Values: []*string{
                    aws.String("available"),
                },
            },
        },
        MaxResults: aws.Int64(50),
    })

Expected behavior
Reference the godoc?

Screenshots
N/A

Additional context
N/A

Error in CLI syntax: Tag -> Tags

I created an EBS volume from a snapshot, and used the AWS CLI version:

aws ec2 create-volume --volume-type "gp2" --size 80 --availability-zone "us-west-2a" \
 --snapshot-id "snap-00000000000007" --no-encrypted \
 --tag-specifications '[{"ResourceType":"volume", \
  "Tag":[\
    {"Key":"Name","Value":"volume-from-snapshot-of-ec2-prod-buildserv"}\
    ] \
  }]' --region us-west-2

This failed, because "Tag" needed to be "Tags".

Also, the other language versions show the tags except TerraForm:

resource "aws_ebs_volume" "ec211111111" {
    type = "gp2"
    size = 80
    availability_zone = "us-west-2a"
    snapshot_id = "snap-000000000007"
    encrypted = false
    tags {
        
    }

}

Version 0.3.29

Love the program! Thanks!

AWS Console fails (requests blocked) while ACR is recording

Describe the bug
Some AWS console pages break when ACR is recording messages.

Related Mapping
sqs.createqueue

Related Language
N/A

To Reproduce
Steps to reproduce the behavior:

  1. Go to 'https://console.aws.amazon.com/sqs/home?region=us-west-2'
  2. Click on 'Get Started Now'
  3. Type a queue name, e.g. 'testqueue'
  4. Click on 'Quick Create Queue'
  5. See error message from AWS Console and AWS Console Recorder plugin
  6. On the ACR console, the SQS request has the region in the queue name:
    aws sqs create-queue --queue-name "us-west-2" --region us-east-1

Expected behavior
A queue should be created.

Screenshots
aws-console-recorder

Additional context

  • This may be related to the uMatrix ad blocker plugin I have installed, but I've disabled it for all AWS console pages.
  • This is on Chrome 79.0.3945.117 (Official Build) (64-bit), on Ubuntu 18.04.3 LTS.
  • AWS Console Recorder Version 0.3.31

ec2.RunInstances output syntax errors

Describe the bug
Python and CLI output for ec2.RunInstances contains syntax errors.

Related Mapping
ec2.RunInstances

Related Language
CLI and Python tested

To Reproduce
Steps to reproduce the behavior:

  1. Start recorder
  2. From EC2 Dashboard click Launch Instance
  3. Select default amazon linux 2 ami
  4. Select default t2.micro instance type
  5. Change default VPC/subnet to anything else.
  6. Next, Next, Next
  7. On security group setup page, choose any existing security group.
  8. Next next finish through wizard

Expected behavior
Expectation is generated commands can be run without modification. This is not the case. For example, using above repro steps generates the following CLI command:

ec2 run-instances --image-id "ami-009d6802948d06e52" --count 1 --key-name "mykey" --instance-type "t2.micro" --placement '{"Tenancy":"default"}' --monitoring '{"Enabled":false}' --enable-api-termination --instance-initiated-shutdown-behavior "stop" --credit-specification '{"CpuCredits":"standard"}' --no-ebs-optimized --block-device-mappings '[{"DeviceName":"/dev/xvda","Ebs":{"VolumeSize":8,"DeleteOnTermination":true,"VolumeType":"gp2"}}]' --region us-east-1

Attempting to run this command results in:

Error parsing parameter '--block-device-mappings': Expected: '=', received: ''' for input:
'[{DeviceName:/dev/xvda,Ebs:{VolumeSize:8,DeleteOnTermination:true,VolumeType:gp2}}]'

The correctly formatted command is:

aws ec2 run-instances --image-id "ami-009d6802948d06e52" --count 1 --key-name "mykey" --instance-type "t2.micro" --placement Tenancy="default" --monitoring Enabled=false --enable-api-termination --instance-initiated-shutdown-behavior "stop" --credit-specification CpuCredits="standard" --no-ebs-optimized --block-device-mappings DeviceName="/dev/xvda",Ebs={VolumeSize=8,DeleteOnTermination=true,VolumeType="gp2"} --region us-east-1

Similar incorrect syntax is seen with python. Recorder output:

import boto3
ec2_client = boto3.client('ec2', region_name='us-east-1')
response = ec2_client.run_instances(
    ImageId='ami-009d6802948d06e52',
    MaxCount=1,
    MinCount=1,
    KeyName='mykey',
    InstanceType='t2.micro',
    Placement={
        Tenancy='default'
    },
    Monitoring={
        Enabled=False
    },
    DisableApiTermination=False,
    InstanceInitiatedShutdownBehavior='stop',
    CreditSpecification={
        CpuCredits='standard'
    },
    EbsOptimized=False,
    BlockDeviceMapping=[
        {
            DeviceName='/dev/xvda',
            Ebs={
                VolumeSize=8,
                DeleteOnTermination=True,
                VolumeType='gp2'
            }
        }
    ]
)

Running this results in:

python test.py
  File "test.py", line 10
    Tenancy='default'
           ^
SyntaxError: invalid syntax

...due to improperly formatted parameter section. The correct formatting is:

response = ec2_client.run_instances(
    ImageId='ami-009d6802948d06e52',
    MaxCount=1,
    MinCount=1,
    KeyName='mykey',
    InstanceType='t2.micro',
    Placement={
        'Tenancy':'default'
    },
    Monitoring={
        'Enabled':False
    },
    DisableApiTermination=False,
    InstanceInitiatedShutdownBehavior='stop',
    CreditSpecification={
        'CpuCredits':'standard'
    },
    EbsOptimized=False,
    BlockDeviceMappings=[
        {
            'DeviceName':'/dev/xvda',
            'Ebs':{
                'VolumeSize':8,
                'DeleteOnTermination':True,
                'VolumeType':'gp2'
            }
        }
    ]
)

creating a route table not working

Service
EC2

API Calls / Actions
create-route-table

CloudFormation Resource Types
AWS::EC2::RouteTable

Additional Information
seems that creating a route-table is not implemented, did not find it as recorded action either in CFN nor in the CLI or Boto3 output
Could also be a bug.

outdated CFN ENI properties

Describe the bug
When I record the creation of a EC2 instance and I try to create another instance via CFN, it fails because of incorrect attribute names.

Related Mapping
AWS::EC2::Instance

Related Language
CFN yml

To Reproduce
Steps to reproduce the behavior:

  1. Create EC2 instance with maximum possible defaults

Expected behavior
EC2 instance can be created via CFN template.
I'd be glad to submit a PR, but you need to create a contribute.md and give a short explanation of how to test the changes locally.

Additional context
The current ENI attribute names are very different nowadays: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-network-iface-embedded.html
Generated EC2 template is here:
ec21e273c3: Type: "AWS::EC2::Instance" Properties: ImageId: "ami-08935252a36e25f85" InstanceType: "t2.micro" Tenancy: "default" Monitoring: false DisableApiTermination: false InstanceInitiatedShutdownBehavior: "stop" CreditSpecification: CPUCredits: "standard" EbsOptimized: false UserData: "eXVtIHVwZGF0ZSAteQp5dW0gaW5zdGFsbCAteSBodHRwZDI0IHBocDcwIG15c3FsNTYtc2VydmVyIHBocDcwLW15c3FsbmQKc2VydmljZSBodHRwZCBzdGFydA==" NetworkInterfaces: - DeviceIndex: 0 Description: "Primary network interface" AssociatePublicIpAddress: true DeleteOnTermination: true SubnetId: "subnet-0878c5e4f8289df48" Groups: - "sg-073d0770cb507b308" Ipv6AddressCount: 0 BlockDeviceMappings: - DeviceName: "/dev/xvda" Ebs: VolumeSize: 8 DeleteOnTermination: true VolumeType: "gp2"

AWS::ECS::TaskDefinition Encountered Unsupported Property image

Describe the bug
When recording a ECS TaskDefinition the property in ContainerDefinitions are recorded starting with a lowercase letter but need to be a uppercase first letter

Related Mapping
AWS::ECS::TaskDefinition

Related Language
Cloudfomration (maybe others as well)

To Reproduce
Steps to reproduce the behavior:

  1. Record AWS::ECS::TaskDefinition
  2. check CF script
  3. see image instead of Image (other as well)

Expected behavior
should be Image

Invalid property "VPCOptions: VPCId" in AWS ES VPC cloudformation template

Hi, first of all I want to congratulate you for the awesome product, it is truly amazing!
While using it I just found the following small bug when recording a creation of an AWS-ES domain on cloudformation:
Describe the bug
template will have the invalid property
VPCOptions: VPCId: "vpc-id123" ...

as in:

resource:
        Type: "AWS::Elasticsearch::Domain"
        Properties:
(...)
            VPCOptions: 
                VPCId: "vpc-ID123"
                SubnetIds: (...)

**Related Mapping**
Type: "AWS::Elasticsearch::Domain"

**To Reproduce**
follow standard procedure to create a VPC ES domain:
see: https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-vpc.html

**Expected behavior**
as described in the official documentation, this option is not valid:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticsearch-domain.html?shortFooter=true#cfn-elasticsearch-domain-vpcoptions
 
is not a showstopper whatsoever, just in case you have the same issue somewhere else. Thanks again for this contribution!!!

AWS BackUp

I am trying to get a template of how to create all the required AWS Backup resources (Backup plan, backup vault, etc).

It would be of great assistance if creating the above resources using the console, while having the Console Recorder running, would create the cloudformation template for these resources, so it can be automated when recreating them.

Thanks in advance.

Support for capturing create and modify of SG and rules

Is your feature request related to a problem? Please describe.
We use a third party software to lock Security Group modifications and I wanted to capture and create the cross referenced SGs with the CLI, so that it can be done quickly enough to avoid the third party system picking up the original SG and then treat it as modified.

Describe the solution you'd like
Capture the SG create and update commands

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Only describe-security-groups calls were captured

Missing Tags in output for CFN

Describe the bug
Noticed that Tags seems not get transferred into the output for CFN.
Output for boto3 has the tags included.
example for CFN:
Resources:
ec20062a21:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: "10.1.0.0/16"
InstanceTenancy: "default"
Resources:
ec20062a21:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: "10.1.0.0/16"
InstanceTenancy: "default"

ec2eb57359:
    Type: "AWS::EC2::InternetGateway"

Same resource in Boto3:
response = ec2_client.create_vpc(
CidrBlock='10.1.0.0/16',
AmazonProvidedIpv6CidrBlock=False,
InstanceTenancy='default'
)
response = ec2_client.create_tags(
Resources=[
'vpc-0192bf319969f9c91'
],
Tags=[
{
'key': 'Name',
'value': 'LabVPC'
}
]
)
response = ec2_client.describe_vpcs()
response = ec2_client.describe_account_attributes()
response = ec2_client.describe_account_attributes()
response = ec2_client.describe_internet_gateways()
response = ec2_client.create_internet_gateway(
)
response = ec2_client.create_tags(
Resources=[
'igw-0ad4370f4e7ebd627'
],
Tags=[
{
'key': 'Name',
'value': 'LabIGW'
}
]
)
Seems that create tags is not mapped.

CFN AWS::EC2::InternetGateway mapping invalid

Describe the bug
When I create a new IGW with default settings (just setting the name), there's an empty properties attribute that also gets created. CFN will reject this because "Properties 'null' values are not allowed in templates"

Related Mapping
AWS::EC2::InternetGateway

Related Language
CFN yml

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
The empty properties does not appear in the template.

Indent Size

First, thanks for writing this awesome tool. It's been needed for a long time, and I really appreciate it.

For a feature request, could we consider setting a "Number of Spaces" for the Cloudformation output? Small issue, but I generally prefer 2 spaces to 4 in my CFN.

Thanks again!

RoutTableId get not tranferred into a !Ref <value>

Describe the bug
I see the following in the output for CFN:
ec2a5a8cd6:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref ec20062a21

ec29da2ac9:
    Type: "AWS::EC2::Route"
    Properties:
        RouteTableId: "rtb-0e63402f072e08d8e"
        DestinationCidrBlock: "0.0.0.0/0"
        NatGatewayId: !Ref ec2369c7ba

ec21c10d70:
    Type: "AWS::EC2::SubnetRouteTableAssociation"
    Properties:
        SubnetId: !Ref ec2b18debb
        RouteTableId: "rtb-0e63402f072e08d8e"

Going through the whole template it is just the RouteTableId what did not get transferred into a !Ref.

Option to hide ec2_client.describe_ actions

Problem

Boto3 output comes out very noisy if performing an action through a wizard. Most of the generated python script is nothing but ec2_client.describe_ commands.

This is especially noticeable with autocomplete form fields, such as the "add tags" form when launching an EC2 instance - a new ec2_client.describe_tags command is added for each keystroke.

Requested solution

Settings parameter or on-page filter to hide/remove all ec2_client.describe_ actions.

Currently just doing this in a text editor, but would be a very helpful quality-of-life feature.

CloudWatch Alarm generation isnt properly defining dimensions.

Describe the bug
CloudWatch Alarm recording is incorrectly defining the dimensions for the metric being captured when the metric has multiple dimensions. The code being generated does have the correct count of dimensions, but all appear to be defined as the same one.

Related Mapping
cloudwatch putMetricAlarm

Related Language
Affects all generated languages

To Reproduce
Steps to reproduce the behavior:

  1. In the AWS console navigate to CloudWatch -> Alarms
  2. Click "Create Alarm"
  3. Choose a Metric that has multiple dimensions, e.g. ApplicationELB: Per AppELB, per AZ, per TG Metrics
  4. Enter the relevant Alarm criteria.. name, description, actions etc.
  5. Save the alarm

Expected behavior
The generated code should have multiple unique dimensions defined, one for each dimension based on the metric chosen for the alarm.

Screenshots
CloudFormation example:

cloudwatch6ef73f1:
    Type: "AWS::CloudWatch::Alarm"
    Properties:
        Dimensions: 
          - 
            Name: "TargetGroup"
            Value: "targetgroup/XXXXX-XXXXX-XXXXX/aaaabbbbccccdddd"
          - 
            Name: "TargetGroup"
            Value: "targetgroup/XXXXX-XXXXX-XXXXX/aaaabbbbccccdddd"
          - 
            Name: "TargetGroup"
            Value: "targetgroup/XXXXX-XXXXX-XXXXX/aaaabbbbccccdddd"
        OKActions: []

Additional context
It appears that the code isn't iterating the dimensions and simply using the same dimension each time for the qty of dimensions defined.

Add verbiage around privacy/data flow

I love this idea, but I think it would help alleviate security concerns if there was clear verbiage/architecture diagram about how data is captured in the README.

CFN template for "launch vpc wizard" invalid

Describe the bug
The CFN template created for the "launch vpc wizard" is invalid.

Steps to reproduce the behavior:
0. Start the recorder

  1. Go to VPC
  2. Click on 'Launch vpc wizard'. Don't change anything, just give the name and click "create VPC"
  3. Stop recording and try to create a CFN stack with the output.
  4. CFN cannot create the stack because of the same id for all the resources.

Expected behavior
The CFN stack is successfully created.

Additional context
I've attached the invalid cfn template that was generated by the recorder.
console_cfn.txt

ec2.runInstances BlockDeviceMapping "NoDevice"

Describe the bug
aws ec2 run-instances is generating "NoDevice" block device mappings for every letter of the alphabet. The generated commands are technically correct, so this may be a feature request to suppress "NoDevice" mappings, rather than a bug :)

Related Mapping
ec2.runInstances

Related Language
CLI and Python tested.

To Reproduce
Steps to reproduce the behavior:

  1. Start recorder
  2. Run through EC2 Launch Instances wizard, accept all defaults.
  3. Stop recorder.

Expected behavior
Expect Block Device Mappings for existent devices only. Instead we get:

aws ec2 run-instances --image-id "ami-0f4c7e570f044b46f" --count 1 --key-name "xxxxxx" --instance-type "t2.micro" --placement "{\"Tenancy\":\"default\"}" --monitoring "{\"Enabled\":false}" --enable-api-termination --instance-initiated-shutdown-behavior "stop" --credit-specification "{\"CpuCredits\":\"standard\"}" --no-ebs-optimized --block-device-mappings "[{\"DeviceName\":\"/dev/sda1\",\"Ebs\":{\"VolumeSize\":30,\"DeleteOnTermination\":true,\"VolumeType\":\"gp2\"}},{\"DeviceName\":\"xvdca\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcb\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcc\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcd\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdce\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcf\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcg\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdch\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdci\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcj\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdck\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcl\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcm\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcn\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdco\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcp\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcq\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcr\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcs\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdct\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcu\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcv\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcw\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcx\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcy\",\"NoDevice\":\"\"},{\"DeviceName\":\"xvdcz\",\"NoDevice\":\"\"}]" --network-interfaces "[{\"DeviceIndex\":0,\"Description\":\"Primary network interface\",\"DeleteOnTermination\":true,\"SubnetId\":\"subnet-xxxxxx\",\"Groups\":[\"sg-xxxxxxxx\"],\"Ipv6AddressCount\":0}]" --region us-east-1

or

response = ec2_client.run_instances(
    ImageId='ami-0f4c7e570f044b46f',
    MaxCount=1,
    MinCount=1,
    KeyName='xxxxxx',
    InstanceType='t2.micro',
    Placement={
        'Tenancy': 'default'
    },
    Monitoring={
        'Enabled': False
    },
    DisableApiTermination=False,
    InstanceInitiatedShutdownBehavior='stop',
    CreditSpecification={
        'CpuCredits': 'standard'
    },
    EbsOptimized=False,
    BlockDeviceMappings=[
        {
            'DeviceName': '/dev/sda1',
            'Ebs': {
                'VolumeSize': 30,
                'DeleteOnTermination': True,
                'VolumeType': 'gp2'
            }
        },
        {
            'DeviceName': 'xvdca',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcb',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcc',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcd',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdce',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcf',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcg',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdch',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdci',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcj',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdck',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcl',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcm',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcn',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdco',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcp',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcq',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcr',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcs',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdct',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcu',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcv',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcw',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcx',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcy',
            'NoDevice': ''
        },
        {
            'DeviceName': 'xvdcz',
            'NoDevice': ''
        }
    ],
    CapacityReservationSpecification={
        'CapacityReservationPreference': 'open'
    },
    NetworkInterfaces=[
        {
            'DeviceIndex': 0,
            'Description': 'Primary network interface',
            'DeleteOnTermination': True,
            'SubnetId': 'subnet-xxxxxxx',
            'Groups': [
                'sg-xxxxxxx'
            ],
            'Ipv6AddressCount': 0
        }
    ]
)

All parameters not captured on ec2.RunInstances

Describe the bug
Recorder is not capturing all parameters entered during recording.

Related Mapping
ec2.RunInstances

Related Language
Only CLI and Python tested

To Reproduce
Steps to reproduce the behavior:

  1. Start recorder
  2. From EC2 Dashboard click Launch Instance
  3. Select default amazon linux 2 ami
  4. Select default t2.micro instance type
  5. Change default VPC/subnet to anything else.
  6. Next, Next, Next
  7. On security group setup page, choose any existing security group.
  8. Next next finish through wizard

Expected behavior
Generated CLI/SDK commands should contain non-default VPC/subnet and security group parameters. They do not.

Using above steps, generated CLI command is as follows. It does not contain a --subnet-id parameter or a --security-group-ids parameter as expected:

aws ec2 run-instances --image-id "ami-009d6802948d06e52" --count 1 --key-name "mykey" --instance-type "t2.micro" --placement '{"Tenancy":"default"}' --monitoring '{"Enabled":false}' --enable-api-termination --instance-initiated-shutdown-behavior "stop" --credit-specification '{"CpuCredits":"standard"}' --no-ebs-optimized --block-device-mappings '[{"DeviceName":"/dev/xvda","Ebs":{"VolumeSize":8,"DeleteOnTermination":true,"VolumeType":"gp2"}}]' --region us-east-1

Recorder-generated python command is as follows. Likewise, it does not contain SubnetId or SecurityGroupIds arguments as expected:

response = ec2_client.run_instances(
    ImageId='ami-009d6802948d06e52',
    MaxCount=1,
    MinCount=1,
    KeyName='mykey',
    InstanceType='t2.micro',
    Placement={
        Tenancy='default'
    },
    Monitoring={
        Enabled=False
    },
    DisableApiTermination=False,
    InstanceInitiatedShutdownBehavior='stop',
    CreditSpecification={
        CpuCredits='standard'
    },
    EbsOptimized=False,
    BlockDeviceMapping=[
        {
            DeviceName='/dev/xvda',a
            Ebs={
                VolumeSize=8,
                DeleteOnTermination=True,
                VolumeType='gp2'
            }
        }
    ]
)

Identifying which HTTP requests are API calls automatically

Is your feature request related to a problem? Please describe.
I'm finding that AWS keeps changing their console, which breaks AWS Console Recorder (ACR).

Describe the solution you'd like
It might help if ACR detected requests that had json in the request/response body, and if it didn't have a rule for deciphering it, at least tell the user about it.

Describe alternatives you've considered
The current system of reporting APIs that are broken is labor-intensive and probably doesn't scale.

Additional context
I love this tool! It was recommended to me by AWS Premium Support. However, it's frustrating when the API call I need is the one that doesn't get recorded due to some small change by AWS.

Thanks!

Strange behavior for References to Logical IDs (CFN)

Describe the bug
I see a strange behavior regarding values for references to logical id's. I have created a simple VPC with 2 subnets and Routing tables etc. This is an example what I got for the CFN subnet resource:
ec2733a861:
Type: "AWS::EC2::VPCGatewayAttachment"
Properties:
VpcId: !Ref ec28471949
InternetGatewayId: !Sub "${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec282e532e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2eb0ae6e.Ipv6CidrBlocks}${ec2320da02}"

I see this entries with Ipv6CidrBlocks at nearly every resource.
I also don't understand why the recorder is using !Sub function and not !Ref.

Cloudformation IAM Role missing managed policies

Describe the bug
I created an IAM Role with managed policies but the generated CFN did not contain them.

Related Mapping
iam:AttacheRoleToPolicy

Related Language
Cloudformation

To Reproduce
Steps to reproduce the behavior:

  1. Go to IAM in console
  2. Create a role with managed policies
  3. Observe generated CFN

Expected behavior
Expected CFN like this:

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
  Generator: "console-recorder"
Description: ""
Resources:
  DevDataUser:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "DevDataUser"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: "Allow"
            Action:
              - "sts:AssumeRole"
            Principal:
              AWS: "XXXXXXXXXX"
            Condition:
              Bool:
                aws:MultiFactorAuthPresent: "true"
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
        - arn:aws:iam::aws:policy/AmazonDynamoDBReadOnlyAccess
        - arn:aws:iam::aws:policy/AmazonRedshiftReadOnlyAccess
        - arn:aws:iam::aws:policy/AmazonRDSReadOnlyAccess

Actual behaviour
Got CFN like this:

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
    Generator: "console-recorder"
Description: ""
Resources:
        iamdea1b3b:
            Type: "AWS::IAM::Role"
            Properties:
            RoleName: "DevDataUser"
            AssumeRolePolicyDocument: 
                    Version: "2012-10-17"
                    Statement: 
                      - 
                            Effect: "Allow"
                            Action: "sts:AssumeRole"
                            Principal: 
                                AWS: "XXXXXXXXXX"
                            Condition: 
                                Bool: 
                                    aws:MultiFactorAuthPresent: "true"

Additional context

Interestingly, the SDK code all use the AttachRolePolicy method after creating the role. I did not test them but they look good. I suppose this is a nuance of Cloudformation that i imagine is a little harder to capture than with the REST APIs

Allocating a new EIP not recorded

Describe the bug
allocating a new EIP is not recorded, neither in CF, nor in CLI or Boto3; hove not checked the others.
Problem exists also when creating a new NATGW and selecting "Create New EIP" during creation process

Related Mapping
missing: AWS::EC2::EIP / aws ec2 allocate-address

Related Language
If applicable to a specific language output, e.g. JavaScript

To Reproduce
Steps to reproduce the behavior:
Start recording

  1. Go to 'VPC'
  2. Click on 'Elastic IPs'
  3. Click on 'Allocate new address'
  4. Stop recording
  5. check Recorder Dashboard; no entry under CF, only describe entries under Boto3

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

can not capture the console operation on edit role->policy in json form.

Describe the bug

When I was operation in console IAM->Roles->Summary->Policy name -> Edit policy and edit the json then save the policy json.

Console Recorder for AWS only gives me
import boto3

iam_client = boto3.client('iam', region_name='xxx')

response = iam_client.list_policies()

Key operation was missing

https://console.aws.amazon.com/iam/home?#/roles/rolename$jsonEditor?policyName=policyname&section=permissions&step=edit

Related Mapping
If applicable to a specific type, e.g. ec2.DescribeInstances

Related Language
If applicable to a specific language output, e.g. JavaScript

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

AWS GovCloud Support

Problem
The recorder does not appear to work in GovCloud.

Solution
I would benefit from the solution working in GovCloud.

Alternatives
None

ec2.RunInstances python "BlockDeviceMapping" typo

Describe the bug
Python ec2.RunInstances "BlockDeviceMappings" parameter incorrectly generating as "BlockDeviceMapping".

Related Mapping
ec2.RunInstances

Related Language
Python

To Reproduce
Steps to reproduce the behavior:

  1. Start recorder
  2. From EC2 Dashboard click Launch Instance
  3. Select default amazon linux 2 ami
  4. Select default t2.micro instance type
  5. Change default VPC/subnet to anything else.
  6. Next, Next, Next
  7. On security group setup page, choose any existing security group.
  8. Next next finish through wizard

Expected behavior
boto3 ec2 client run_instances() argument is "BlockDeviceMappings" (see https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.run_instances). Recorder is currently outputting "BlockDeviceMapping".

For example:

response = ec2_client.run_instances(
    ImageId='ami-009d6802948d06e52',
    MaxCount=1,
    MinCount=1,
    KeyName='mykey',
    InstanceType='t2.micro',
    Placement={
        Tenancy='default'
    },
    Monitoring={
        Enabled=False
    },
    DisableApiTermination=False,
    InstanceInitiatedShutdownBehavior='stop',
    CreditSpecification={
        CpuCredits='standard'
    },
    EbsOptimized=False,
    BlockDeviceMapping=[
        {
            DeviceName='/dev/xvda',
            Ebs={
                VolumeSize=8,
                DeleteOnTermination=True,
                VolumeType='gp2'
            }
        }
    ]
)

Cloudformation output produced for AWS::Glue::Connection is incorrect

Describe the bug
When recording a Glue Connection, the resulting Cloudformation text is malformed.

Related Mapping
glue.createConnection

Related Language
Cloudformation

To Reproduce

  1. Create a Glue Connection from

Expected behavior
This is what I get currently:

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
    Generator: "console-recorder"
Description: ""
Resources:
    glue2a9e07f:
        Type: "AWS::Glue::Connection"
        Properties:
            ConnectionInput: 
              - 
                action: "com.amazonaws.console.glue.awssdk.shared.context.AWSGlueContext.createConnection"
                parameters: 
                  - 
                    connectionInput: 
                        physicalConnectionRequirements: 
                            availabilityZone: "us-east-1d"
                            subnetId: "<redacted>"
                            securityGroupIdList: 
                              - "<redacted>"
                        connectionType: "JDBC"
                        description: ""
                        name: "my-connection-name"
                        connectionProperties: 
                            USERNAME: "<redacted>"
                            PASSWORD: "<redacted>"
                            JDBC_CONNECTION_URL: "jdbc:<redacted>"
                            JDBC_ENFORCE_SSL: "true"

It should be something like:

AWSTemplateFormatVersion: "2010-09-09"
Description: ""

Resources:
  GlueAuroraCrawler:
    Type: "AWS::Glue::Connection"
    Properties:
      ConnectionInput:
        PhysicalConnectionRequirements:
          AvailabilityZone: "us-east-1d"
          SubnetId: "<redacted>"
          SecurityGroupIdList:
            - "<redacted>"
        ConnectionType: "JDBC"
        Description: ""
        Name: "my-connection-name"
        ConnectionProperties:
          USERNAME: "<redacted>"
          PASSWORD: "<redacted>"
          JDBC_CONNECTION_URL: "jdbc:<redacted>"
          JDBC_ENFORCE_SSL: "true"
      CatalogId: !Ref AWS::AccountId

Screenshots
n/a

Additional context
Having a look at the Boto3 code, it looks like this might not be a Cloudformation output problem, but rather how you're parsing the Glue API call results.

If you're accepting PRs, I can take a stab at fixing this one.

Apply linting to the CFN templates through cfn-lint

Idea behind this would be to enforce the strongly typed values for certain resources.
At present the following are put as Int rather than Strings
RDS:
Type: "AWS::RDS::DBInstance"
Properties:
AllocatedStorage: 100
BackupRetentionPeriod: 7
Port: 3306

Could be possibly a bug between cfn-lint and the Cloudformation JSON spec.

Cloudfront support

Thanks for the amazing tool. I have tried recording my cloudfront but nothing seems to happen. Is cloudfront currently supported?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.