Terraform module to create resources on AWS to forward logs/metrics to Datadog 🇺🇦
Home Page: https://registry.terraform.io/modules/terraform-aws-modules/datadog-forwarders/aws
License: Apache License 2.0
No
The RDS enhanced metrics forwarder module doesn't create the Lambda trigger for sending metrics from the RDSOSMetrics CloudWatch log group.
Add a the trigger as either a subscription filter on the log group or a trigger on the Lambda function.
We've had to Terraform the trigger/filter separately, but there's no obvious reason why it can't be included in this module.
Basically step 11 on the installation docs onwards:
https://docs.datadoghq.com/integrations/amazon_rds/?tab=enhanced#installation
module doesn't return lambda_arn
Error: Unsupported attribute
on datadog/rds_enhanced_monitoring.tf line 27, in output "debug":
27: value = module.datadog_rds_enhanced_monitoring_forwarder.lambda_arn
This value does not have any attributes.
The number of tags that can be attached to an s3 bucket object is 10 (for other resources it is 50) Documentation
So, we have an error:
Error: error updating tags: error setting resource tags (s-dd-forwarder-dev/aws-dd-forwarder-3.41.0.zip): BadRequest: Object tags cannot be greater than 10 status code: 400, request id: TE64412FP9X2M6RM, host id: QshfEqkYz3YB1fcBPsmuqZ1jfFP3a3JegL4W5b+YPqD7Vck3FwZ8nvIewvdg4hrb+HTUdZS3q1I1bjEiZ/ZQBA== on .terraform/modules/datadog_log_forwarder/modules/log_forwarder/main.tf line 131, in resource "aws_s3_bucket_object" "this": 131: resource "aws_s3_bucket_object" "this" {
We need all these tags in other objects and we can't reduce them.
Could you, please, reduce or delete the number of tags in the module that can be attached to this object?
Is your feature request related to a problem? Please describe.
Using AWS default encryption key is generally discouraged, best practice is to use customer-managed keys, therefore it would be great if the module would allow specifying a customer-managed key. Ref: tfsec.dev/docs/aws/s3/enable-bucket-encryption
Describe the solution you'd like
Ability to provide a CMK for bucket encryption
Describe alternatives you've considered
None
Additional context
From convo on #9
Shouldn't the GetObject permission be <bucket>/* rather than just <bucket>?
Upgraded from 4.1.0 to 5.0.1 and the log forwarding function stopped working. Played with Python version - 3.7 and 3.9 fail, 3.8 works fine. Datadog doc clearly says to use Python 3.8 https://docs.datadoghq.com/logs/guide/forwarder/
"Since version 3.49.0 the Lambda function has been updated to require Python 3.8. If upgrading an older forwarder installation to 3.49.0 or above, ensure the AWS Lambda function is configured to use Python 3.8"
module "datadog_forwarders" {
source = "terraform-aws-modules/datadog-forwarders/aws"
version = "5.0.1"
log_forwarder_name = var.name
create_rds_em_forwarder = false
create_vpc_fl_forwarder = false
dd_api_key_secret_arn = data.aws_secretsmanager_secret.datadog_api_key.arn
kms_alias = aws_kms_alias.datadog.name
log_forwarder_reserved_concurrent_executions = var.log_forwarder_reserved_concurrent_executions
tags = var.tags
}
The function fails with:
Runtime.ImportModuleError: Unable to import module 'lambda_function': cannot import name '_rand' from 'ddtrace.internal' (/var/task/ddtrace/internal/__init__.py)
Default Python version should be changed to 3.8
Describe the bug
Hey there, so I recently attempted an upgrade of this module to the latest version (3.7.0) from 3.1.1 and the forwarder Lambda started failing and spitting out these logs:
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': cannot import name '_rand' from 'ddtrace.internal' (/var/task/ddtrace/internal/__init__.py)
I was previously pinned to version 3.1.1 of this module and 3.40.0 of the Datadog forwarder and tried to update to 3.7.0 and 3.44.0, respectively. Though if I pin to 3.7.0 and forwarder version 3.40.0, I see this in the plan and it still breaks:
~ resource "aws_lambda_function" "this" {
~ architectures = [
+ "x86_64",
]
id = "preprod-datadog-logs-forwarder"
~ last_modified = "2022-03-23T17:50:35.000+0000" -> (known after apply)
~ runtime = "python3.7" -> "python3.9"
tags = {
"DD_FORWARDER_VERSION" = "3.40.0"
"Env" = "preprod"
}
# (20 unchanged attributes hidden)
# (2 unchanged blocks hidden)
}
I'm guessing this has to do with the architecture and Python runtime updates introduced in v3.5.0, and I found your GitHub issue with the exact same log over here: DataDog/datadog-lambda-python#88 ...but I'm not clear on how to remediate and I'd love any insight or suggestions you might have as it seems you've dug into it pretty deep.
To Reproduce
This is my old configuration:
module "datadog_log_forwarder" {
source = "clowdhaus/datadog-forwarders/aws"
version = "3.1.1"
dd_api_key = ""
dd_api_key_secret_arn = var.api_key_secret_arn
kms_alias = var.kms_key_alias
log_forwarder_version = "3.40.0"
log_forwarder_name = "${var.environment_name}-datadog-logs-forwarder"
log_forwarder_environment_variables = {
DD_TAGS = "env:${local.dd_env}"
}
bucket_name = "my-bucket-name"
create_vpc_fl_forwarder = false
}
And this is the new configuration that broke, though the log_forwarder_version can be set to 3.40.0 and the error still occurs:
module "datadog_log_forwarder" {
source = "clowdhaus/datadog-forwarders/aws"
version = "3.7.0"
dd_api_key = ""
dd_api_key_secret_arn = var.api_key_secret_arn
kms_alias = var.kms_key_alias
log_forwarder_version = "3.44.0"
log_forwarder_name = "${var.environment_name}-datadog-logs-forwarder"
log_forwarder_environment_variables = {
DD_TAGS = "env:${local.dd_env}"
}
bucket_name = "my-bucket-name"
create_vpc_fl_forwarder = false
}
Expected behavior
The forwarder Lambda function continues to run without error
Thanks!
This might be halfway between a "bug" (because of the description) and a feature request at the same time...
The s3_log_bucket_arns variable in https://github.com/terraform-aws-modules/terraform-aws-datadog-forwarders/tree/master/modules/log_forwarder has a description that mentions:
S3 log buckets for forwarder to read and forward logs to Datadog
...yet, looking through the code (and also trying it out), is it possible that it does not take care of adding the actual forwarding of logs to Datadog?
I was expecting to see a aws_s3_bucket_notification resource, which I believe would have taken care of this part, but there does not seem to be one? Or might I be missing something?
Module version [Required]: 6.0.1
Terraform version: 1.8.3
+ provider registry.terraform.io/datadog/datadog v3.39.0
+ provider registry.terraform.io/hashicorp/aws v5.49.0
+ provider registry.terraform.io/hashicorp/null v3.2.2
+ provider registry.terraform.io/hashicorp/random v3.6.1
Lambda function being aware of new logs in S3 bucket and forwarding those to Datadog. Lambda logs/metrics showing activity. New logs appearing in Datadog.
S3 logs are not getting forwarded by the lambda function. Lambda function logs are empty. Nothing appearing in Datadog logs.
Is your feature request related to a problem? Please describe.
Can you please add support for AWS Provider 4.0+
Describe the solution you'd like
Support for AWS Provider 4.0+
Additional context
S3 Bucket only module used in this module already released support for AWS Provider 4.x - https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/releases/tag/v3.0.0
Describe the bug
The list function is no longer present in Terraform 0.15.
To Reproduce
Just try to use this module with Terraform 0.15
Expected behavior
Working with the same behavior as in Terraform 0.14
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
╷
│ Error: Error in function call
│
│ on .terraform/modules/datadog_forwarder.this_s3_bucket/outputs.tf line 3, in output "this_s3_bucket_id":
│ 3: value = element(concat(aws_s3_bucket_policy.this.*.id, aws_s3_bucket.this.*.id, list("")), 0)
│
│ Call to function "list" failed: the "list" function was deprecated in
│ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to
│ write a literal list.
╵
╷
│ Error: Error in function call
│
│ on .terraform/modules/datadog_forwarder.this_s3_bucket/outputs.tf line 8, in output "this_s3_bucket_arn":
│ 8: value = element(concat(aws_s3_bucket.this.*.arn, list("")), 0)
│
│ Call to function "list" failed: the "list" function was deprecated in
│ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to
│ write a literal list.
╵
╷
│ Error: Error in function call
│
│ on .terraform/modules/datadog_forwarder.this_s3_bucket/outputs.tf line 13, in output "this_s3_bucket_bucket_domain_name":
│ 13: value = element(concat(aws_s3_bucket.this.*.bucket_domain_name, list("")), 0)
│
│ Call to function "list" failed: the "list" function was deprecated in
│ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to
│ write a literal list.
╵
╷
│ Error: Error in function call
│
│ on .terraform/modules/datadog_forwarder.this_s3_bucket/outputs.tf line 18, in output "this_s3_bucket_bucket_regional_domain_name":
│ 18: value = element(concat(aws_s3_bucket.this.*.bucket_regional_domain_name, list("")), 0)
│
│ Call to function "list" failed: the "list" function was deprecated in
│ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to
│ write a literal list.
╵
╷
│ Error: Error in function call
│
│ on .terraform/modules/datadog_forwarder.this_s3_bucket/outputs.tf line 23, in output "this_s3_bucket_hosted_zone_id":
│ 23: value = element(concat(aws_s3_bucket.this.*.hosted_zone_id, list("")), 0)
│
│ Call to function "list" failed: the "list" function was deprecated in
│ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to
│ write a literal list.
╵
╷
│ Error: Error in function call
│
│ on .terraform/modules/datadog_forwarder.this_s3_bucket/outputs.tf line 28, in output "this_s3_bucket_region":
│ 28: value = element(concat(aws_s3_bucket.this.*.region, list("")), 0)
│
│ Call to function "list" failed: the "list" function was deprecated in
│ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to
│ write a literal list.
╵
╷
│ Error: Error in function call
│
│ on .terraform/modules/datadog_forwarder.this_s3_bucket/outputs.tf line 33, in output "this_s3_bucket_website_endpoint":
│ 33: value = element(concat(aws_s3_bucket.this.*.website_endpoint, list("")), 0)
│
│ Call to function "list" failed: the "list" function was deprecated in
│ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to
│ write a literal list.
╵
╷
│ Error: Error in function call
│
│ on .terraform/modules/datadog_forwarder.this_s3_bucket/outputs.tf line 38, in output "this_s3_bucket_website_domain":
│ 38: value = element(concat(aws_s3_bucket.this.*.website_domain, list("")), 0)
│
│ Call to function "list" failed: the "list" function was deprecated in
│ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to
│ write a literal list.
╵
Is your feature request related to a problem? Please describe.
We're using hashicorp vault to store all secrets so the required kms key is not suitable.
Describe the solution you'd like
make kms keys optional
I updated AWS provider version for my Terraform project from 3.75.2 to 4.9.0 and warning started appearing.
│ Warning: Argument is deprecated │ │ with module.my_module.module.datadog_log_forwarder.module.datadog_log_forwarder.module.this_s3_bucket.aws_s3_bucket.this, │ on .terraform/modules/my_module.datadog_log_forwarder.datadog_log_forwarder.this_s3_bucket/main.tf line 14, in resource "aws_s3_bucket" "this": │ 14: resource "aws_s3_bucket" "this" { │ │ Use the top-level parameter object_lock_enabled and the aws_s3_bucket_object_lock_configuration resource instead
There are newer versions of terraform-aws-s3-bucket module, starting with 3.1.0, that fixed said warning.
Module version [Required]: 4.1.1
Terraform version: 1.2.4
Provider version(s): 4.9.0
Steps to reproduce the behavior:
Are you using workspaces? No.
Have you cleared the local cache (see Notice section above)? Yes.
List steps in order that led up to the issue you encountered:
terraform init terraform plan terraform apply -auto-approve
Terraform plan and apply executed without the warning.
Terraform plan and apply executed with the described warning.
The change here:
... seems to grant the lambdas permission to kms:Decrypt on any KMS key, which seems bad from a security perspective?
Thanks!
Is this functionality available in the AWS provider for Terraform? See CHANGELOG.md, too.
I encountered an error with Terraform AWS provider v4 -> v5 upgrade.
Change that Terraform want to make:
module.datadog_log_forwarder.aws_s3_object.this[0] will be updated in-place
~ resource "aws_s3_object" "this" {
- acl = "private" -> null
id = "aws-dd-forwarder-3.71.0.zip"
tags = {
"DD_FORWARDER_VERSION" = "3.71.0"
}
# (14 unchanged attributes hidden)
}Error for previous plan:
╷
│ Error: putting S3 object ACL: MissingSecurityHeader: Your request was missing a required header
│ status code: 400
│
│ with module.datadog_log_forwarder.aws_s3_object.this[0],
│ on .terraform/modules/datadog_log_forwarder.datadog_log_forwarder/modules/log_forwarder/main.tf line 131, in resource "aws_s3_object" "this":
│ 131: resource "aws_s3_object" "this" {
│
╵
Set acl = "private" for aws_s3_object.this[0]
The number of tags that can be attached to an s3 bucket object is 10 AWS Documentation
For other AWS resources, this limit is 50.
So, in the case when we use more than 10 tags for the whole module we have an error
It doesn't depend on terraform. It's AWS limit.
Module version [Required]:
"terraform-aws-modules/datadog-forwarders/aws//modules/log_forwarder"
version = "4.1.1"
Terraform version:
Terraform v0.14.11
Provider version(s):
├── provider[registry.terraform.io/hashicorp/aws] ~> 4.0
├── provider[registry.terraform.io/opsgenie/opsgenie] ~> 0.6
├── provider[terraform.io/builtin/terraform]
├── module.tags
│ └── provider[registry.terraform.io/hashicorp/aws] >= 3.38.0
├── module.datadog_log_forwarder
│ ├── provider[registry.terraform.io/hashicorp/aws] >= 4.0.0
│ ├── provider[registry.terraform.io/hashicorp/null] >= 3.0.0
│ └── module.this_s3_bucket
│ └── provider[registry.terraform.io/hashicorp/aws] >= 3.75.0
├── module.labels
└── module.opsgenie_service
├── provider[registry.terraform.io/opsgenie/opsgenie] ~> 0.6
└── provider[registry.terraform.io/hashicorp/null]
Providers required by state:
provider[registry.terraform.io/hashicorp/aws]
provider[registry.terraform.io/hashicorp/null]
provider[terraform.io/builtin/terraform]
Steps to reproduce the behavior:
You need also to add more than 10 tags to the variable tags ;
variable "tags" {
description = "A map of tags to use on all resources"
type = map(string)
default = {
"Name.1"="a"
"Name.2"="a"
"Name.3"="a"
"Name.4"="a"
"Name.5"="a"
"Name.6"="a"
"Name.7"="a"
"Name.8"="a"
"Name.9"="a"
"Name.10"="a"
"Name.11"="a"
}
}
module "datadog_log_forwarder" {
source = "terraform-aws-modules/datadog-forwarders/aws//modules/log_forwarder"
version = "4.1.1"
dd_api_key = var.dd_api_key
dd_site = "datadoghq.eu"
forwarder_version = "3.41.0"
bucket_name = "s-dd-forwarder-${var.env}"
kms_key_arn = var.cmk_arn
role_tags = var.role_tags
tags = var.tags
environment_variables = { DD_TAGS = "env:cloud-${var.env}" }
}
The bucket zip object created without any tags
Now the bucket zip object tags as other AWS resources, but terraform isn't applied if the tags count more than 10
Error: error updating tags: error setting resource tags (s-dd-forwarder-dev/aws-dd-forwarder-3.41.0.zip): BadRequest: Object tags cannot be greater than 10
status code: 400, request id: TE64412FP9X2M6RM, host id: QshfEqkYz3YB1fcBPsmuqZ1jfFP3a3JegL4W5b+YPqD7Vck3FwZ8nvIewvdg4hrb+HTUdZS3q1I1bjEiZ/ZQBA==
on .terraform/modules/datadog_log_forwarder/modules/log_forwarder/main.tf line 131, in resource "aws_s3_bucket_object" "this":
131: resource "aws_s3_bucket_object" "this" {
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1
tfsec reports the following issues:
Please provide a clear and concise description of the issue you are encountering, and a reproduction of your configuration (see the examples/* directory for references that you can copy+paste and tailor to match your configs if you are unable to copy your exact configuration). The reproduction MUST be executable by running terraform init && terraform apply without any further changes.
If your request is for a new feature, please use the Feature request template.
Before you submit an issue, please perform the following first:
.terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/terraform initModule version [Required]:
Terraform version:
Terraform v1.4.3-dev
on darwin_amd64
module "datadog_forwarders" {
source = "terraform-aws-modules/datadog-forwarders/aws"
kms_alias = "alias/datadog"
dd_api_key_secret_arn = data.aws_secretsmanager_secret_version.datadog_api_key.arn
create_log_forwarder = true
create_rds_em_forwarder = true
create_vpc_fl_forwarder = false
vpc_id = "XXXXX"
}
Steps to reproduce the behavior:
terraform apply
The log forwarder should be created and should forward logs
Log forwarder throws error:
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': cannot import name '_rand' from 'ddtrace.internal' (/var/task/ddtrace/internal/__init__.py)
Traceback (most recent call last):
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.