aws-cloudformation / aws-cloudformation-resource-providers-rds Goto Github PK

I am using AWS Backup to manage RDS cluster backups and using CDK to manage Aurora DB cluster.

I added a tag to RDS Aurora DB Cluster but the stack fails with AWS Backup conflicts

RDS cluster xyz is associated with the following AwsBackupRecoveryPointArn: arn:aws:backup:us-east-1:000000000000:recovery-point:continuous:cluster-fwqvlm34vzrxkdelm7stdduw6q-f50fc296. The BackupRetentionPeriod can be blank, or you can use the current value, 10. For more details, see the AWS Backup documentation. (Service: Rds, Status Code: 400, Request ID: 123)"

AddTagsToResource should have been sufficient to add this tag however after looking in CloudTrail, the update handler invokes ModifyDbCluster containing backupRetentionPeriod with a default value of 1 . Example:

{
  "eventVersion": "1.08",
  "userIdentity": {
    "type": "AssumedRole",
    "principalId": "ABC12345:AWSCloudFormation",
    "arn": "arn:aws:sts::000000000000:assumed-role/cdk-hnb659fds-cfn-exec-role-000000000000-us-east-1/AWSCloudFormation",
    "accountId": "000000000000",
    "accessKeyId": "ABC123",
    "sessionContext": {
      "sessionIssuer": {
        "type": "Role",
        "principalId": "ABC123",
        "arn": "arn:aws:iam::000000000000:role/cdk-hnb659fds-cfn-exec-role-000000000000-us-east-1",
        "accountId": "000000000000",
        "userName": "cdk-hnb659fds-cfn-exec-role-000000000000-us-east-1"
      },
      "webIdFederationData": {},
      "attributes": {
        "creationDate": "2024-02-20T17:50:16Z",
        "mfaAuthenticated": "false"
      }
    },
    "invokedBy": "cloudformation.amazonaws.com"
  },
  "eventTime": "2024-02-20T17:50:16Z",
  "eventSource": "rds.amazonaws.com",
  "eventName": "ModifyDBCluster",
  "awsRegion": "us-east-1",
  "sourceIPAddress": "cloudformation.amazonaws.com",
  "userAgent": "cloudformation.amazonaws.com",
  "errorCode": "InvalidParameterValueException",
  "errorMessage": "RDS cluster xyz is associated with the following AwsBackupRecoveryPointArn: arn:aws:backup:us-east-1:000000000000:recovery-point:continuous:cluster-fwqvlm34vzrxkdelm7stdduw6q-f50fc296. The BackupRetentionPeriod can be blank, or you can use the current value, 10. For more details, see the AWS Backup documentation.",
  "requestParameters": {
    "dBClusterIdentifier": "xyz",
    "applyImmediately": true,
    "backupRetentionPeriod": 1,
    "dBClusterParameterGroupName": "default.aurora-postgresql15",
    "cloudwatchLogsExportConfiguration": {
      "enableLogTypes": [],
      "disableLogTypes": []
    },
    "allowMajorVersionUpgrade": false,
    "copyTagsToSnapshot": true,
    "allowEngineModeChange": false
  },
  "responseElements": null,
  "requestID": "123",
  "eventID": "123",
  "readOnly": false,
  "eventType": "AwsApiCall",
  "managementEvent": true,
  "recipientAccountId": "000000000000",
  "eventCategory": "Management"
}

I believe this mismatch is the cause of the issue.

The workaround is to add retention to the RDS cluster directly on the CDK app/CFN template:

backup: {
        retention: Duration.days(10),
}

  "DatabaseB269D8BB": {
   "Type": "AWS::RDS::DBCluster",
   "Properties": {
    "BackupRetentionPeriod": 10,

Can you take a look at this?

RDS WS API team owns the resource

the createGlobalCluster is only takes arn of dbCluster: https://docs.aws.amazon.com/cli/latest/reference/rds/create-global-cluster.html
need to add a way to translate dbClusterIdentifier to arn in the create handler

Name of the resource

AWS::RDS::DBInstance

Resource name

No response

Description

We want to provision RDS on Outposts with the option to store backup locally on Outpost. However, it seems that there is no option to enable that via CloudFormation. The feature is available for CLI, SDK and console.

We cannot update Backup Target after creation so updating the resource after provision via CFN is out of question.

Provision RDS on Outposts:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-on-outposts.creating.html

CloudFormation reference:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbinstance.html

Other Details

Related to aws-cloudformation/cloudformation-coverage-roadmap#1984

AWS::RDS::Integration

Aurora zero-ETL with Redshift was just released, but it's not possible to use this via CloudFormation.

Acceptance Criteria:

• AWS::RDS::Integration resource created with all options available in the API.

need to add 60s propagation delay to avoid InvalidDBClusterStateFault exception

According to the AWS documentation there should be "SecretArn" attribute on DBCluster resource: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html#aws-resource-rds-dbcluster-return-values

It is a bit worrisome that its description is "Property description not available." but shouldn't this be available in CloudFormation?

When ManageMasterUserPassword is se to true the RDS will create new secret in secret manager which isn't referencable anywhere in CloudFormation template, which is where the RDS cluster SecretArn attribute comes in

{
	"Resources": {
		"DatabaseCluster": {
			"Type": "AWS::RDS::DBCluster",
			"Properties": {
				"ManageMasterUserPassword": true,
				// other properties
			}
		}
	},
	"Outputs": {
		"ClusterSecretArn": {
			"Value": {
				"Fn::Sub": "${DatabaseCluster.SecretArn}"
			},
			"Export": {
				"Name": "cluster-secret-arn"
			}
		}
	},
}

We can configure “Retain Automated Backups” for RDS DBInstance using the DeleteAutomatedBackups property in CloudFormation. However, the property is not supported in RDS DBCluster CloudFormation resource even though the DeleteDBCluster API call has DeleteAutomatedBackups as a request parameter.

We request that the 'DeleteAutomatedBackups' property be added to the 'AWS::RDS::DBCluster' resource.

(This issue duplicates aws-cloudformation/cloudformation-coverage-roadmap#192, as I'm interpreting the contribution guidelines in this way; if that is incorrect, please let me know.)

This attribute is needed for some specific circumstances, and cannot currently be retrieved through GetAttr.

Hi,

I just wanted to make you aware of aws-cloudformation/cloudformation-coverage-roadmap#1014, which seems to affect this resource provider. I am re-reporting this here, since there has been no interaction (maybe I was reporting to the wrong repo).

1. need to change the properties restriction in template.
   1. if GlobalClusterIdentifier exist,
      1. check if global cluster has members
      2. if global cluster has members, cannot specify username and password
   2. action needed: because we cannot check if global cluster has members from template, allow either to have or don’t have user name and password if globalClusterIdentifier exists

We are aware that CloudFormation & RDS ServiceTeam is actively working on migration of RDS resource within CFN platform. We facing different CloudFormation Drift Detection false-positives issues, which I'd like to link here as separate comments. By doing so, this issue acts as an overview / master-issue. Once all linked issues are closed, we/AWS should close this issue here as well.

Name of the resource
AWS::RDS::DBInstance

Resource name
No response

Description
RDS Oracle has come up with support for converting non-CDB databases to CDB databases.

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/oracle-multitenant.html#oracle-cdb-converting

CloudFormation documentation offers "oracle-ee-cdb" as "Engine" value in the AWS::RDS::DBInstance resource and also Update requires: [Some interruptions] gives an impression that update to "Engine" is possible.

However if we try and update "Engine" from "oracle-ee" to "oracle-ee-cdb" we get "Resource is immutable" error.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbinstance.html#cfn-rds-dbinstance-engine

Since, support for non-CDB to CDB is allowed through RDS Console/CLI; AWS CloudFormation should support this well.

RDS WS API team will own the resource

Hi Team,

Is there any update yet regarding this pull request?

https://github.com/aws-cloudformation/aws-cloudformation-resource-providers-rds/pull/537/files

PR #163 implemented support for non-Aurora multi-AZ DB clusters, which can now be created through CloudFormation by setting the Engine to either mysql or postgres, however the schema and public CloudFormation documentation page hasn't been updated to reflect this. Although the new attributes required for Multi-AZ clusters are present in the schema, the descriptions for existing attributes (such as Engine) still indicate that only Aurora clusters are supported.

If this support for Multi-AZ DB clusters is official and supported, can the schema and docs be updated?

Name of the resource

AWS::RDS::DBInstance

Resource name

No response

Description

RDS Oracle has come up with support for Converting the single-tenant configuration to multi-tenant.

Since, support for enable multi-tenant is allowed through RDS Console/CLI; AWS CloudFormation should support this well.

Documentation

https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-rds-oracle-supports-oracle-multitenant/?nc1=h_ls

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/oracle-single-tenant-converting.html

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbinstance.html

currently RDS API v2 does not include sourceRegion - CreateDBCluster. This property must be added once RDS fixes their API

When we delete dbcluster that is member of global cluster, we need to remove it from global cluster first:
https://www.google.com/search?client=firefox-b-e&q=remove-from-global-cluster
action needed:

1. first check if globalClusterIdentifier != null, if yes, trigger removeFromGlobalCluster first
2. current removeFromGlobalCluster only accept arn, need to do use describeDBCluster to get the cluster arn and pass the arn to removeFromGlobalCluster

The documentation says that the Engine property is optional. Then what is the default value and why is the engine parameter required in AWS CLI and terraform?

Cross posting this issue from aws-cloudformation/cloudformation-coverage-roadmap#281

Expected Behavior:

According to the documentation, when creating a AWS::RDS::DBCluster, if the Port property is not specified it should default to port 3306 if the Engine property is set to aurora or 5432 if it's set to aurora-postgresql.

Actual Behavior:

Regardless of the value set for the 'Engine' property, the Port property defaults to 3306.

this issue is an internal tracker of this issue until resource is published

As captured in aws-cloudformation/cloudformation-coverage-roadmap#1426, the AWS::RDS::DBCluster ignores the EnableCloudwatchLogsExports property when creating a clone.

Aurora-Redshift zero-ETL now supports table-level filtering via DataFilter attribute. Customers should be able to use the new filter using CFN.

Not sure if this is the correct place to submit this.

Trying to set IAM DB authentication to true on the already existing cluster via cloudFormation, but when setting EnableIAMDatabaseAuthentication: true in the cloudFormation seems to have no effect on this. Am I reading the documentation correct? I was looking at this page AWS::RDS::DBCluster

Given the following abridged template, given the IsDatabaseImportExportEnabled condition is true, the resource is created/updated with only 1 associated role for s3Import

Key Facts

• RDS: Aurora PostgreSQL 12.12

Reason Why This Is Wrong:

• The template I have created, expects the same IAM role to be added for both s3Import and s3Export features.
• The template executed via CFN without errors and there was no drift detected.
• However s3Export did not work.
• As the cluster is Amazon Aurora with PostgreSQL compatibility I have viewed documentation at https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.Authorizing.IAM.AddRoleToDBCluster.html.
• When I realised that the role was only added for s3Import feature, via checking the DB Cluster in the AWS Console, I attempted to add the same role for s3Export however the console rejected this and gave me an error message saying that the same role has already been added.
• I believe in this case the template should have been rejected or at least, the update on the DB Cluster should have failed.
• I note the document linked above did not mention that the same role can only be used for one feature.

Workaround:

• I have created a 2nd IAM role and associated it for the s3Export feature

DatabaseCluster:
  Type: AWS::RDS::DBCluster
  DependsOn:
    - DatabaseClusterParameterGroup
    - DatabaseInstanceParameterGroup
  Properties:
    AssociatedRoles:
      - !If
        - IsDatabaseImportExportEnabled
        - FeatureName: s3Import
          RoleArn: !GetAtt DatabaseImportExportRole.Arn
        - !Ref AWS::NoValue
      - !If
        - IsDatabaseImportExportEnabled
        - FeatureName: s3Export
          RoleArn: !GetAtt DatabaseImportExportRole.Arn
        - !Ref AWS::NoValue
    BackupRetentionPeriod: !Ref DatabaseBackupRetentionPeriod
    DatabaseName: postgres
    DBClusterParameterGroupName: !Ref DatabaseClusterParameterGroup
    DBSubnetGroupName: !Ref DatabaseSubnetGroup
    EnableCloudwatchLogsExports:
      - postgresql
    Engine: aurora-postgresql
    EngineVersion: '12.12'
    MasterUsername: !Sub '{{resolve:secretsmanager:${DatabaseMasterUsernameSecret}:SecretString::}}'
    MasterUserPassword: !Sub '{{resolve:secretsmanager:${DatabaseMasterPasswordSecret}:SecretString::}}'
    Port: 5432
    StorageEncrypted: true
    VpcSecurityGroupIds:
      - !Ref DatabaseSecurityGroup

When cloning an RDS Aurora MySQL cluster from one account to another that is using a customer parameter group name Cloudformation omits the Parameter Groups that are provided in the template. The following error occurs.

Resource handler returned message: "The cluster parameter group name is required when the source cluster uses a custom parameter group.

The error is InvalidParameterCombinationException because the parameter group settings from the template are not handed to the RDS service.

To fix this the Cloudformation service would need to provide the Parameter Group Name settings set in the AWS::RDS::DBCluster resource to the RDS API call for RestoreDBClusterToPointInTime.

Must of this is taken directly from the CloudFormation roadmap issue here: aws-cloudformation/cloudformation-coverage-roadmap#529 submitted by @abatkin.

I am not clear if this would actually require a restart or not, the API documentation doesn't note a restart is automatically applied but the console has a screenshot stating:

"If you modify a DB cluster to enable Kerberos authentication, reboot the DB cluster after making the change."

AWS::RDS::DBCluster should support the Domain (and likely DomainIAMRoleName) for Aurora database clusters (at least Postgres, but I think MySQL too).

The Domain field is a Directory ID from AWS Managed Microsoft Active Directory, likely a String, unless you want to support this as a first-class CloudFormation Type.

The documentation for these API parameters are here:

https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBCluster.html
https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_ModifyDBCluster.html
Note that it should be possible to update this value on an existing cluster without replacing it.

These parameters should not be confused with the Domain and DomainIAMRoleName on AWS::RDS::DBInstance as this aspect of an Aurora database appears to be controlled at the Cluster and not the Instance level (plus those two attributes are clearly documented in both the CloudFormation and the RDS documentation as being valid only for Microsoft SQL Server and Oracle DB).

With the feature Global Databases, we will need to add GlobalClusterIdentifier property to DBCluster entity to support creation of cluster in the global cluster. This property can be optional when customer create DBCluster.
Related document: https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBCluster.html

RDS WS API team owns the resource

When attempting to deploy a simple AWS::RDS::DBCluster resource the stack creation fails with the following error

Resource handler returned message: "Tag keys cannot start with the reserved prefix "aws:".

Even more curiously, this seems to only be happening in the ap-northeast-1 region, but not eu-west-1. (will update case as issue is detected in other regions). When deploying an RDS Cluster resource to ap-northeast-1, the error is observed. However, when deploying to eu-west-1 creates just fine.

Curiously, it seems that the two regions are running different schemas of this particular resource type as well. let's spare the verbose details for now, but comparing the output of the following commands will reveal significantly different schemas.

aws cloudformation describe-type --type-name AWS::RDS::DBCluster --type RESOURCE --region eu-west-1
aws cloudformation describe-type --type-name AWS::RDS::DBCluster --type RESOURCE --region ap-northeast-1

Replication

Deploying this template to the ap-northeast-1 region will fail at creation with the above-mentioned error

Parameters:
  MasterUserPassword:
    Type: String
    NoEcho: true
    Default: 'password123'
  MasterUsername:
    Type: String
    Default: MyName123
  DatabaseName:
    Type: String
    Default: testdb

Resources:
  MyDbCluster:
    Type: AWS::RDS::DBCluster
    Properties:
      DatabaseName: !Ref DatabaseName
      Engine: postgres
      EngineVersion: "13.4"
      MasterUsername: !Ref MasterUsername
      MasterUserPassword: !Ref MasterUserPassword
      AllocatedStorage: 100
      DBClusterInstanceClass: "db.m5d.large"
      StorageType: "io1"
      Iops: 3000
    DeletionPolicy: Delete
  MyDbInstance1:
    Type: AWS::RDS::DBInstance
    Properties:
      DBClusterIdentifier: !Ref MyDbCluster
      DBInstanceClass: "db.m5d.large"
      Engine: postgres
      EngineVersion: "13.4"

The following CreateDBCluster event can be seen.

{
    "eventVersion": "1.08",
    "userIdentity": {
        "type": "IAMUser",
        "principalId": "XXXXXXXXXXX",
        "arn": "arn:aws:iam::00000000000:user/yooser",
        "accountId": "00000000000",
        "accessKeyId": "XXXXXXXXXXXXXXXXXXXXX",
        "userName": "yooser",
        "sessionContext": {
            "sessionIssuer": {},
            "webIdFederationData": {},
            "attributes": {
                "creationDate": "2022-07-22T12:15:18Z",
                "mfaAuthenticated": "false"
            }
        },
        "invokedBy": "cloudformation.amazonaws.com"
    },
    "eventTime": "2022-07-22T12:15:26Z",
    "eventSource": "rds.amazonaws.com",
    "eventName": "CreateDBCluster",
    "awsRegion": "ap-northeast-1",
    "sourceIPAddress": "cloudformation.amazonaws.com",
    "userAgent": "cloudformation.amazonaws.com",
    "errorCode": "InvalidParameterValueException",
    "errorMessage": "Tag keys cannot start with the reserved prefix \"aws:\".",
    "requestParameters": {
        "databaseName": "testdb",
        "engineVersion": "13.4",
        "allocatedStorage": 100,
        "backupRetentionPeriod": 1,
        "dBClusterIdentifier": "sandbox-mydbcluster-klxxxxxx4txqs",
        "engine": "postgres",
        "masterUserPassword": "****",
        "dBClusterInstanceClass": "db.m5d.large",
        "tags": [
            {
                "value": "sandbox",
                "key": "aws:cloudformation:stack-name"
            },
            {
                "value": "arn:aws:cloudformation:ap-northeast-1:00000000000:stack/sandbox/ef377e50-09b7-11ed-b47a-xxxxxxxxxx",
                "key": "aws:cloudformation:stack-id"
            },
            {
                "value": "MyDbCluster",
                "key": "aws:cloudformation:logical-id"
            }
        ],
        "port": 3306,
        "storageType": "io1",
        "masterUsername": "MyName123",
        "iops": 3000
    },
    "responseElements": null,
    "requestID": "xxxxxx-xxxx-xxx-xxxxx-xxxxxxxx",
    "eventID": "xxxxxx-xxxx-xxx-xxxxx-xxxxxxxx",
    "readOnly": false,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "00000000000",
    "eventCategory": "Management"
}

Conclusion

It seems that the Cloudformation service is not expecting the service-managed tags with aws-reserved prefixes. Would you kindly help determining if a bugfix is required here, and if possible, offer potential workarounds in the meantime.

Please let me know if further information is required and I will be happy to provide.

AWS::RDS::DBClusterEndpoint - can be created via API, but not via CloudFormation

Scope of request

Add support for Aurora custom cluster endpoints.

Expected behavior

Have the same options as the API.
During update the resource should always require replacement. This is because updates to custom endpoints cause gaps in DNS resolution of the endpoint updated.

https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateIntegration.html

The maximum length for the integration is 63, not 64.

It seems the current implementation explicitly skips the final snapshot on DBCluster delete invokation: https://github.com/aws-cloudformation/aws-cloudformation-resource-providers-rds/blob/master/aws-rds-dbcluster/src/main/java/software/amazon/rds/dbcluster/Translator.java#L159

The former implementation, in contrast, supports this feature. Sounds like something we should support.

When updating parameter groups the UpdateHandler detects the "pending-reboot" state and reboots the instance.

The problem with this is that it ends up rebooting all instances in a cluster at the same time, meaning availability of the cluster is compromised. Ideally the affected instances would be rebooted sequentially, maximising availability of the cluster.

As a workaround, we are:

• Separating cloud formation stacks containing parameter groups and instances
• Updating the parameter group stack when changes are needed
• Manually performing the reboot (with a failover for the writer) sequentially

Apologies if this is a very dumb question, and I don't know where else to ask, but I am very new to Cloudformation. Moreover, I am not easygoing with GitHub as well. I was looking for rds instance automated backup replication and in AWS documentation I was unable to find the property "AutomaticBackupReplicationRegion".
In this repository, very recently the property "AutomaticBackupReplicationRegion" is added . However I am not sure how to use this repository as I just create the yaml templates using aws syntax and when I added this property in my AWS::RDS::DBInstance resource, it is throwing error.