andrewfarley / aws-automated-daily-instance-ami-snapshots Goto Github PK

Idea from @charterresources from #1

Thank you for your contribution @AndrewFarley
Everything works perfect without error but for some unknown reason it only creates images for 7 out of 19 instances.
All the instances have the tag "Backup" and retention tag with a value of "11"
When I run a test, it creates images of the same 7 instances over and over again. If I remove the tag "Backup" from these 7 instances, the test run will create 1 out of remaining 12 instances with the tag "Backup". If I remove the tag from this instance too, the test will create another one of the remaining 11 instances.
Please help.

Here is the Log output:

START RequestId: 7b1b6c6f-7307-11e8-a0a3-a5e096e1e21f Version: $LATEST
Scanning region: us-east-1
Scanning for instances with tags (backup,Backup)
Found 7 instances to backup...
Instance: i-#############
Name: @@@@@@
Time: 11 days
AMI: ami-#####
Instance: i-#############
Name: @@@@@@@
Time: 11 days
AMI: ami-#####
Instance: i-#############
Name: @@@@@
Time: 11 days
AMI: ami-#####
Instance: i-#######
Name: @@@@@
Time: 11 days
AMI: ami-######
Instance: i-#############
Name: @@@@@
Time: 11 days
AMI: ami-######
Instance: i-###########
Name: @@@
Time: 11 days
AMI: ami-#######
Instance: i-############
Name: @@@
Time: 11 days
AMI: ami-########
Scanning for AMIs with tags (CESRBackupInstanceRotater)
END RequestId: 7b1b6c6f-7307-11e8-a0a3-a5e096e1e21f
REPORT RequestId: 7b1b6c6f-7307-11e8-a0a3-a5e096e1e21f Duration: 3154.76 ms Billed Duration: 3200 ms Memory Size: 128 MB Max Memory Used: 46 MB

Idea from @charterresources from #1

Add the option of shutting down specific EC2 instances that may require clean snapshots (i.e an additional tag of shutdown) and once completed start them back up.

Hello, is there any good way to implement the functions of multiple backup cycles?
For example, the first cycle is daily backup, and the expired AMI is deleted every week.
The second cycle is weekly backup, deleting expired AMIs every month.
The third cycle is monthly backup, deleting expired AMIs every year.
Looking forward to your reply.

Reported via email...

Failure trying to create image or tag image.  See/report exception below

Traceback (most recent call last):

  File "/var/task/lambda_function.py", line 104, in backup_tagged_instances_in_region

    Tags=instance['Tags']

  File "/var/runtime/botocore/client.py", line 314, in _api_call

    return self._make_api_call(operation_name, kwargs)

  File "/var/runtime/botocore/client.py", line 612, in _make_api_call

    raise error_class(parsed_response, operation_name)

botocore.exceptions.ClientError: An error occurred (InvalidParameterValue) when calling the CreateTags operation: Value ( aws:autoscaling:groupName ) for parameter key is invalid. Tag keys starting with 'aws:' are reserved for internal use

Scanning for AMIs with tags (AWSAutomatedDailySnapshots)

This occurs because some instances created with CloudFormation or that are associated to an autoscaler have tags on them prefixed with aws:. We should skip these tags.

The lambda runtime should be updated to something newer than 3.6

• 3.7, 3.8, or 3.9.

July 18, 2022 => end of security updates
Aug 17, 2022  => end of create, update access

Links

• https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html - Runtime deprecation policy section
• https://peps.python.org/pep-0494/
  • 3.6.15 final: 2021-09-04 => last security-only release

Hi Andrew,
Many thanks for your awesome code!!
I would like a similar code but only on volumes. Take a snapshot every week and delete the previous. I am in the Paris region and we don't have this native function.
Could you please help me to modify your code regarding only volumes tagged Backup:true? I'm not interested to create an AMI only a backup of EBS data volume.
Many thanks