opensearch-project / terraform-provider-opensearch Goto Github PK
View Code? Open in Web Editor NEWHome Page: https://registry.terraform.io/providers/opensearch-project/opensearch
License: Apache License 2.0
Home Page: https://registry.terraform.io/providers/opensearch-project/opensearch
License: Apache License 2.0
The opensearch_index
resource does not seem to store or manage the mappings
property.
opensearch
provider with valid connection (e.g. url, credentials, etc) to an existing OpenSearch server with a valid index containing mappings.opensearch_index
resource with a mappingI'm unable to create all the necessary objects by way of Terraform.
Index Patterns, which I think you need for data streams, are not present in the provider.
Though it might be a Kibana concept?
A resource type opensearch_index_pattern
that operates about the same as everything else in the provider.
Combination of local_file
with jsonencode
and null
resource of local_script
to manually curl
the API.
https://www.elastic.co/guide/en/kibana/7.17/index-patterns.html
Hello,
I have 2 questions regarding terraform resource https://github.com/opensearch-project/terraform-provider-opensearch/blob/main/docs/resources/ism_policy_mapping.md
ism_template
in the policy definition can pick automatically the new indexes and it works fine but with existing indexes, I assume the ism_policy_mapping resource should be created.managed_indexes
so that when i update the policy it also reflects the managed indexes?Thanks a lot in advance
When using AWS SSO profiles the provider crashes.
Attempt to apply changes using AWS SSO profiles that rely on sso_session
.
Graceful shutdown
aarch64-darwin
$ uname -a
Darwin bne-nb-ariel 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 arm Darwin
$ terraform -version
Terraform v1.3.9
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.7.0
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
│ Error: Plugin did not respond
│
│ with opensearch_index.test,
│ on opensearch.tf line 10, in resource "opensearch_index" "test":
│ 10: resource "opensearch_index" "test" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵
Stack trace from the terraform-provider-opensearch_v1.0.0 plugin:
panic: profile "Search.Dev.EngineeringAccess" is configured to use SSO but is missing required configuration: sso_region, sso_start_url
goroutine 39 [running]:
github.com/aws/aws-sdk-go/aws/session.Must(...)
github.com/aws/[email protected]/aws/session/session.go:381
github.com/opensearch-project/terraform-provider-opensearch/provider.awsSession({0x1400030e2f0, 0xe}, 0x14000728b40)
github.com/opensearch-project/terraform-provider-opensearch/provider/provider.go:525 +0x340
github.com/opensearch-project/terraform-provider-opensearch/provider.awsHttpClient({0x1400030e2f0, 0xe}, 0x14000728b40, 0x1008daf74?)
github.com/opensearch-project/terraform-provider-opensearch/provider/provider.go:529 +0x38
github.com/opensearch-project/terraform-provider-opensearch/provider.getClient(0x14000728b40)
github.com/opensearch-project/terraform-provider-opensearch/provider/provider.go:303 +0x6d4
github.com/opensearch-project/terraform-provider-opensearch/provider.resourceOpensearchIndexCreate(0x14000877238?, {0x1015a9800?, 0x14000728b40})
github.com/opensearch-project/terraform-provider-opensearch/provider/resource_opensearch_index.go:526 +0x9b0
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0x1017c69c0?, {0x1017c69c0?, 0x140002ec900?}, 0xd?, {0x1015a9800?, 0x14000728b40?})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:330 +0x138
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0x140002b0c40, {0x1017c69c0, 0x140002ec900}, 0x140008385b0, 0x140002f6c80, {0x1015a9800, 0x14000728b40})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:472 +0x714
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0x1400019d8d8, {0x1017c6918?, 0x140002b28c0?}, 0x140002f2230)
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:1021 +0xb5c
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0x1400078f9a0, {0x1017c69c0?, 0x140002ec030?}, 0x140002ee000)
github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:812 +0x38c
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x101740400?, 0x1400078f9a0}, {0x1017c69c0, 0x140002ec030}, 0x1400061a120, 0x0)
github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:385 +0x174
google.golang.org/grpc.(*Server).processUnaryRPC(0x1400027c700, {0x1017c9660, 0x14000318680}, 0x140004a5560, 0x1400078a540, 0x101f33f40, 0x0)
google.golang.org/[email protected]/server.go:1282 +0xb3c
google.golang.org/grpc.(*Server).handleStream(0x1400027c700, {0x1017c9660, 0x14000318680}, 0x140004a5560, 0x0)
google.golang.org/[email protected]/server.go:1619 +0x840
google.golang.org/grpc.(*Server).serveStreams.func1.2()
google.golang.org/[email protected]/server.go:921 +0x88
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/[email protected]/server.go:919 +0x298
Error: The terraform-provider-opensearch_v1.0.0 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.
I believe sso_session
is unsupported by the AWS GoLang SDK.
The current setup is using olivere elastic go client, use official supported OpenSearch go client
The project terraform-provider-opensearch since its being released and managed under opensearch-project org for ensuring long term compatibility it should using official supported OpenSearch go client
Since the project will be released as OpenSearch terraform provider the build and release should be using go client that is managed and fully compatible with OpenSearch. The existing code base uses olivere elastic go client that has more compatibility with elastic (FYI this analysis is based on project README doc)
The developers for contributing, building and testing should be moving forward with OpenSearch go client which could include change of certain client code methods and functionalities.
This would definitely improve the security scope as the OpenSearch go client is an umbrella project of OpenSearch project.
Expected to have some go client methods and functionalities change in code.
NO IMPACT
NO IMPACT
The code should be modified and be using OpenSearch go client that refers olivere elastic go client,
Open for suggestions from the community :)
Coming from the comment #8 (comment), in order to release the provider the rename to OpenSearch and Dashboard has to be done with the codebase.
Also the go client used is https://github.com/olivere/elastic, which mentioned not to use at production, the project should be migrated to use OpenSearch go client https://github.com/opensearch-project/opensearch-go
Resource opensearch_cluster_settings
attempts to apply null
to unset properties, despite them being defaulted and valid in-cluster.
opensearch_cluster_settings
Existing and valid settings should be pulled into Terraform state from the cluster and not overridden unless the property is set in the resource block.
uname -a && terraform -version
Darwin bne-nb-ariel 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 arm Darwin
Terraform v1.5.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.62.0
+ provider registry.terraform.io/hashicorp/external v2.3.1
+ provider registry.terraform.io/hashicorp/local v2.4.0
+ provider registry.terraform.io/hashicorp/null v3.2.1
+ provider registry.terraform.io/hashicorp/random v3.5.1
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
+ provider registry.terraform.io/phillbaker/elasticsearch v2.0.4
Opensearch - by design for SaaS - prevents managing certain cluster settings.
Ideally, settings that aren't available for a given OpenSearch version shouldn't be configurable in the resource block.
But implementing that might be a lot of boilerplate.
Need implementation of Snapshot Management API https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore/
Support the Snapshot Management API in this provider: https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/snapshots/sm-api/#create-or-update-a-policy
Create snapshots using direct API calls or using the Dashboards UI.
no
A new dashboard object cannot be created if the assigned index doesn't exist. The terraform apply will fail saying plugin crash. However, if you check the indices in Opensearch, the given index actually got created from the failed deployment. If I rerun the Terraform plan and apply again, things would work as expected.
.kibana_<hash>_<tenant_name>
.kibana_<hash>_<tenant_name>
index already got created from the first failed deployment.│ 61: resource "opensearch_dashboard_object" "index_patterns" {
│
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
Stack trace from the terraform-provider-opensearch_v1.0.0 plugin:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xef475a]
goroutine 147 [running]:
github.com/opensearch-project/terraform-provider-opensearch/provider.elastic7CreateIndexIfNotExists(0xc00052ee00, {0xc00076f980, 0x15}, {0xc00076f980, 0x15})
github.com/opensearch-project/terraform-provider-opensearch/provider/resource_opensearch_dashboard_object.go:141 +0x21a
github.com/opensearch-project/terraform-provider-opensearch/provider.resourceOpensearchDashboardObjectCreate(0xc0003fe600, {0xf84780?, 0xc000507400})
github.com/opensearch-project/terraform-provider-opensearch/provider/resource_opensearch_dashboard_object.go:100 +0x10f
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0x14565e8?, {0x14565e8?, 0xc0005366c0?}, 0xd?, {0xf84780?, 0xc000507400?})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:330 +0x178
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc00018b340, {0x14565e8, 0xc0005366c0}, 0xc00062b790, 0xc0003fe480, {0xf84780, 0xc000507400})
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:472 +0x83a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc000438798, {0x1456540?, 0xc0004eb000?}, 0xc00040d2c0)
github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:1021 +0xdaa
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc0003f5c20, {0x14565e8?, 0xc000525ec0?}, 0xc0001f60e0)
github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:812 +0x515
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x111a540?, 0xc0003f5c20}, {0x14565e8, 0xc000525ec0}, 0xc000467a40, 0x0)
github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:385 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000170a80, {0x1459308, 0xc000445a00}, 0xc0008b70e0, 0xc00043b290, 0x1bdfee0, 0x0)
google.golang.org/[email protected]/server.go:1282 +0xccf
google.golang.org/grpc.(*Server).handleStream(0xc000170a80, {0x1459308, 0xc000445a00}, 0xc0008b70e0, 0x0)
google.golang.org/[email protected]/server.go:1619 +0xa1b
google.golang.org/grpc.(*Server).serveStreams.func1.2()
google.golang.org/[email protected]/server.go:921 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/[email protected]/server.go:919 +0x28a
Error: The terraform-provider-opensearch_v1.0.0 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.
It seems like the provider is trying to support creating an index pattern with a nonexistent index but it crashes after the index got created. If that is the case, the expected behavior should be the index patterns will be created successfully with a new index.
Operating system, version.
If applicable, add screenshots to help explain your problem.
Add any other context about the problem.
A clear and concise description of the bug.
Steps to reproduce the behavior.
tf as below:
resource "opensearch_index_template" "template_1" {
name = "test"
body = <<EOF
{
"index_patterns": [
"logs*"
],
"template": {
"settings": {
"number_of_shards": 2,
"number_of_replicas": 2
},
"mappings": {
"properties": {}
}
}
}
EOF
}
terraform could apply the change, but can't find the corresponding template in OpenSearch-Index Management-Template menu.
A clear and concise description of what you expected to happen.
Operating system, version.
If applicable, add screenshots to help explain your problem.
Add any other context about the problem.
Found in HEAD commit: 08ca809a1111164e3d8a2fa2d3bfd1ea11700a29
CVE | Severity | Dependency | Type | Fixed in (github.com/Aws/Aws-sdk-go-v1.43.21 version) | Remediation Available | |
---|---|---|---|---|---|---|
CVE-2022-41721 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ | |
CVE-2022-32149 | 7.5 | golang.org/x/text-v0.3.7 | Transitive | N/A* | ❌ |
*For some transitive vulnerabilities, there is no version of direct dependency with a fix. Check the "Details" section below to see if there is a version of transitive dependency where vulnerability is fixed.
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: 08ca809a1111164e3d8a2fa2d3bfd1ea11700a29
Found in base branch: main
A request smuggling attack is possible when using MaxBytesHandler. When using MaxBytesHandler, the body of an HTTP request is not fully consumed. When the server attempts to read HTTP2 frames from the connection, it will instead be reading the body of the HTTP request, which could be attacker-manipulated to represent arbitrary HTTP2 requests.
Publish Date: 2023-01-13
URL: CVE-2022-41721
Base Score Metrics:
Type: Upgrade version
Release Date: 2023-01-13
Fix Resolution: v0.2.0
Library home page: https://proxy.golang.org/golang.org/x/text/@v/v0.3.7.zip
Dependency Hierarchy:
Found in HEAD commit: 08ca809a1111164e3d8a2fa2d3bfd1ea11700a29
Found in base branch: main
An attacker may cause a denial of service by crafting an Accept-Language header which ParseAcceptLanguage will take significant time to parse.
Publish Date: 2022-10-14
URL: CVE-2022-32149
Base Score Metrics:
Type: Upgrade version
Origin: https://www.cve.org/CVERecord?id=CVE-2022-32149
Release Date: 2022-10-14
Fix Resolution: v0.3.8
Currently the terraform provider can not be found on the terraform registry. The installation is super simple if its available on there. Also the documentation is nice to use.
Found in HEAD commit: 4c08dd7a5e25b9e79d8de223f405277bd3f122b8
CVE | Severity | Dependency | Type | Fixed in (github.com/hashicorp/terraform-plugin-sdk/v2-v2.10.0 version) | Remediation Available | |
---|---|---|---|---|---|---|
CVE-2022-27664 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ | |
CVE-2022-30633 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ | |
CVE-2022-41721 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ | |
CVE-2022-32149 | 7.5 | golang.org/x/text-v0.3.7 | Transitive | N/A* | ❌ | |
CVE-2022-28131 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ |
*For some transitive vulnerabilities, there is no version of direct dependency with a fix. Check the "Details" section below to see if there is a version of transitive dependency where vulnerability is fixed.
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: 4c08dd7a5e25b9e79d8de223f405277bd3f122b8
Found in base branch: main
In net/http in Go before 1.18.6 and 1.19.x before 1.19.1, attackers can cause a denial of service because an HTTP/2 connection can hang during closing if shutdown were preempted by a fatal error.
Publish Date: 2022-09-06
URL: CVE-2022-27664
Base Score Metrics:
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: 4c08dd7a5e25b9e79d8de223f405277bd3f122b8
Found in base branch: main
Uncontrolled recursion in Unmarshal in encoding/xml before Go 1.17.12 and Go 1.18.4 allows an attacker to cause a panic due to stack exhaustion via unmarshalling an XML document into a Go struct which has a nested field that uses the 'any' field tag.
Publish Date: 2022-08-10
URL: CVE-2022-30633
Base Score Metrics:
Type: Upgrade version
Origin: https://security-tracker.debian.org/tracker/CVE-2022-30633
Release Date: 2022-05-13
Fix Resolution: go1.17.12,go1.18.4
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: 4c08dd7a5e25b9e79d8de223f405277bd3f122b8
Found in base branch: main
A request smuggling attack is possible when using MaxBytesHandler. When using MaxBytesHandler, the body of an HTTP request is not fully consumed. When the server attempts to read HTTP2 frames from the connection, it will instead be reading the body of the HTTP request, which could be attacker-manipulated to represent arbitrary HTTP2 requests.
Publish Date: 2023-01-13
URL: CVE-2022-41721
Base Score Metrics:
Library home page: https://proxy.golang.org/golang.org/x/text/@v/v0.3.7.zip
Dependency Hierarchy:
Found in HEAD commit: 4c08dd7a5e25b9e79d8de223f405277bd3f122b8
Found in base branch: main
An attacker may cause a denial of service by crafting an Accept-Language header which ParseAcceptLanguage will take significant time to parse.
Publish Date: 2022-10-14
URL: CVE-2022-32149
Base Score Metrics:
Type: Upgrade version
Origin: https://www.cve.org/CVERecord?id=CVE-2022-32149
Release Date: 2022-10-14
Fix Resolution: v0.3.8
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: 4c08dd7a5e25b9e79d8de223f405277bd3f122b8
Found in base branch: main
Uncontrolled recursion in Decoder.Skip in encoding/xml before Go 1.17.12 and Go 1.18.4 allows an attacker to cause a panic due to stack exhaustion via a deeply nested XML document.
Publish Date: 2022-08-10
URL: CVE-2022-28131
Base Score Metrics:
Type: Upgrade version
Origin: https://security-tracker.debian.org/tracker/CVE-2022-28131
Release Date: 2022-03-29
Fix Resolution: go1.17.12,go1.18.4
Found in HEAD commit: ebbbc8c2b225c5ff46ad37b8fd7cce8b4441730a
CVE | Severity | Dependency | Type | Fixed in (github.com/AWS/AWS-sdk-go-v1.43.21 version) | Remediation Possible** | |
---|---|---|---|---|---|---|
CVE-2022-41721 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ | |
CVE-2022-32149 | 7.5 | golang.org/x/text-v0.3.7 | Transitive | N/A* | ❌ |
*For some transitive vulnerabilities, there is no version of direct dependency with a fix. Check the "Details" section below to see if there is a version of transitive dependency where vulnerability is fixed.
**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: ebbbc8c2b225c5ff46ad37b8fd7cce8b4441730a
Found in base branch: main
A request smuggling attack is possible when using MaxBytesHandler. When using MaxBytesHandler, the body of an HTTP request is not fully consumed. When the server attempts to read HTTP2 frames from the connection, it will instead be reading the body of the HTTP request, which could be attacker-manipulated to represent arbitrary HTTP2 requests.
Publish Date: 2023-01-13
URL: CVE-2022-41721
Base Score Metrics:
Type: Upgrade version
Release Date: 2023-01-13
Fix Resolution: v0.2.0
Library home page: https://proxy.golang.org/golang.org/x/text/@v/v0.3.7.zip
Dependency Hierarchy:
Found in HEAD commit: ebbbc8c2b225c5ff46ad37b8fd7cce8b4441730a
Found in base branch: main
An attacker may cause a denial of service by crafting an Accept-Language header which ParseAcceptLanguage will take significant time to parse.
Publish Date: 2022-10-14
URL: CVE-2022-32149
Base Score Metrics:
Type: Upgrade version
Origin: https://www.cve.org/CVERecord?id=CVE-2022-32149
Release Date: 2022-10-14
Fix Resolution: v0.3.8
I'm unable to create all the necessary objects by way of Terraform.
Aliases, which are essential to using dynamic indexes and rollovers, are not present in the provider.
A resource type opensearch_alias
that operates about the same as everything else in the provider.
Combination of local_file
with jsonencode
and null
resource of local_script
to manually curl
the API.
Working with the provider in CICD platform that provide support for AssumeRoleWithWebIdentity can be difficult when the provider does not support it. At the moment the provider uses assume role and if credentials OR profile is not provided, it fails with access denied
Implement support for AssumeRoleWithWebIdentity
in the provider. An example of how would that look like is the AWS provider:
provider "aws" {
assume_role_with_web_identity {
role_arn = "arn:aws:iam::123456789012:role/ROLE_NAME"
session_name = "SESSION_NAME"
web_identity_token_file = "/Users/tf_user/secrets/web-identity-token"
}
}
Hacking up a solution in the CICD where it authenticate to AWS and creates a profile before running terraform
No
I've a opensearch_index_template resource. Applying the plan, it creates the resource correctly on my AWS OpenSearch 2.5 domain. If I run plan/apply again, it keeps saying that this resourse will be updated.
Here's a part of the terraform file:
terraform {
required_providers {
opensearch = {
source = "opensearch-project/opensearch"
version = "1.0.0"
}
}
}
[...]
resource "opensearch_index_template" "index_template" {
for_each = toset(local.apps_file)
name = "${each.key}"
body = jsonencode(
{
index_patterns = [
"${each.key}-*"
]
template = {
mappings = {
properties = {}
}
settings = {
index = {
number_of_shards = "1"
}
}
}
}
)
}
I also defined the "body" part as bellow, and the behavor is the same:
body = <<EOF
{
"index_patterns": [
"${each.key}-*"
],
"template": {
"settings": {
"index":{
"number_of_shards": "1"
}
},
"mappings": {
"properties": {}
}
}
}
EOF
}
After apply, the template is there:
GET _template/xyz
{
"xyz": {
"order": 0,
"index_patterns": [
"xyz-*"
],
"settings": {},
"mappings": {},
"aliases": {}
}
}
And if I run plan/apply again, and again, I always get:
[...]
module.opensearch.opensearch_index_template.index_template["xyz"]: Refreshing state... [id=xyz]
[...]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# module.opensearch.opensearch_index_template.index_template["xyz"] will be updated in-place
~ resource "opensearch_index_template" "index_template" {
~ body = jsonencode(
~ {
+ template = {
+ mappings = {
+ properties = {}
}
+ settings = {
+ index = {
+ number_of_shards = "1"
}
}
}
# (1 unchanged attribute hidden)
}
)
id = "xyz"
name = "xyz"
}
Plan: 0 to add, 1 to change, 0 to destroy.
Nothing to do after the first apply.
I use the for_each loop in order to create a resource per application listed on a file. It creates correctly internal users, roles and role mappings, no issue on these.
Hi, I am currently looking at ways to delete an index and the behavior I am seeing when modifying a opensearch_index resource is very confusing.
aliases
of an existing index, the index gets destroyed then recreated by Terraform, logs saying that the index must be replaced, in which case the index is destroyed and recreated. However, when modifying other attributes such as number_of_replicas
or refresh_interval
, the index gets updated in place and the index is kept.force_destroy
to true
or flicking it from false
to true
, Terraform indicates in the plan that the index is changed, but does not actually destroy the index.Provider configuration not present
. This also happens whether I delete the targeted index manually from the OS dashboard or not; so if I delete the index manually and do not remove the module, the index will get recreated, and if I remove the module, I get that error. I am able to get through this stage by doing a state rm
on the module, but this does not actually delete the index either.Create an index without aliases
nor force_destroy
:
aliases
for the index. The index will get destroyed and recreated.force_destroy
to true
. Nothing happens to the index.Terraform v1.3.5
on linux_amd64
...
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
Context = I am seeing this behavior when trying to delete an index.
What would be the correct way to achieve deleting an index that was created using the terraform provider? Is this supported somehow?
Found in HEAD commit: 1ad1fc85b417b88bf19f6040c9fb3a28adb7c696
CVE | Severity | Dependency | Type | Fixed in (github.com/aws/aws-sdk-go-v1.44.333 version) | Remediation Possible** | |
---|---|---|---|---|---|---|
CVE-2022-41721 | 7.5 | golang.org/x/net-v0.1.0 | Transitive | N/A* | ❌ |
*For some transitive vulnerabilities, there is no version of direct dependency with a fix. Check the "Details" section below to see if there is a version of transitive dependency where vulnerability is fixed.
**In some cases, Remediation PR cannot be created automatically for a vulnerability despite the availability of remediation
[mirror] Go supplementary network libraries
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.1.0.zip
Dependency Hierarchy:
Found in HEAD commit: 1ad1fc85b417b88bf19f6040c9fb3a28adb7c696
Found in base branch: main
A request smuggling attack is possible when using MaxBytesHandler. When using MaxBytesHandler, the body of an HTTP request is not fully consumed. When the server attempts to read HTTP2 frames from the connection, it will instead be reading the body of the HTTP request, which could be attacker-manipulated to represent arbitrary HTTP2 requests.
Publish Date: 2023-01-13
URL: CVE-2022-41721
Base Score Metrics:
Type: Upgrade version
Release Date: 2023-01-13
Fix Resolution: v0.2.0
In order to create releases, we'll need the following set as repository secrets for the github actions (I do not have permission):
The values for these will also need to be generated. The personal access token is generated by Github, but needs to have permissions on this repo. A GPG key needs to be generated and the private key added to that environment variable.
As well, we'll need an account created on the Hashicorp terraform registry, see https://registry.terraform.io/publish/provider.
cc @bbarani
Hi, I just noticed that you started this project.
We started our own opensearch provider recently although api coverage is still small at the moment: https://github.com/Ferlab-Ste-Justine/terraform-provider-opensearch
I'm not sure how far you are on the implementation roadmap, but let us know if we can contribute with anything we have done so far.
I would like to propose the possibility to configure more than one static OpenSearch domain at the same time with a terraform apply.
There are a lot of users who, for example, want to further process a set of results from a terraform module where the static configuration of a provider prevents them from doing so.
However, this will not be possible in terraform for the foreseeable future.
We are building a complete platform with everthing-as-code and terraform in AWS.
This includes a lot of OpenSearch domains (in AWS managed OpenSearch).
The AWS terraform provider does not offer OpenSearch specific configurations.
Based on the created OpenSearch domains (list), we now face the challenge of having to perform user mapping for each individual domain, for example.
This does not work with statically initialized providers, because each domain has its own endpoint.
AWS Request Signing must be supported (already supported).
Resources must be customized to support an optional endpoint (e.g., found via a DataSource).
For example, as a user I want to be able to iterate over my list of OpenSearch domains with for_each
in a resource and pass the endpoint in the resource.
This feature can be implemented as an optional feature in my opinion, so there should be no break in usage.
The feature would significantly expand the provider's purpose and make it more flexible.
It saves (in this particular focus) the use of terragrunt or similar external tools.
How can the AWS OpenSearch Provider be harmonized with the OpenSearch Provider so that there is no break in processing?
Terraform cannot import resource opensearch_cluster_settings
opensearch_cluster_settings
terraform import opensearch_cluster_settings foo
Terraform imports and can view the cluster settings state
$ uname -a && terraform -version
Darwin bne-nb-ariel 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 arm Darwin
Terraform v1.5.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.62.0
+ provider registry.terraform.io/hashicorp/external v2.3.1
+ provider registry.terraform.io/hashicorp/local v2.4.0
+ provider registry.terraform.io/hashicorp/null v3.2.1
+ provider registry.terraform.io/hashicorp/random v3.5.1
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
+ provider registry.terraform.io/phillbaker/elasticsearch v2.0.4
Full trace log, password redacted.
https://gist.github.com/arichtman-srt/bb7188bd4526d9ddbedba7975bdafbc5
Possibly related.
phillbaker/terraform-provider-elasticsearch#288 (comment)
Error: opensearch version 2.5.0 is older than 6.0.0 and is not supported, flavor: 0.
1.Provisioning AWS Opensearch version 2.5.0.
2.Create some resource with terraform.
Example
resource "opensearch_index_template" "log_aws_default" { count = var.environment == "prod" ? 1 : 0 name = "log-aws_default" body = <<EOF { "index_patterns": [ "log-aws-*" ], "order": 0, "settings": { "index": { "number_of_shards": "1", "number_of_replicas": "1" } }, "mappings": { "_source": { "enabled": true } }, "aliases" : { "log-aws": {} } } EOF }
3. Run terraform apply.
Terraform apply all resources successfully.
AWS Opensearch version 2.5.0.
│ Error: opensearch version 2.5.0 is older than 6.0.0 and is not supported, flavor: 0. │ │ with opensearch_ism_policy.log_aws_default[0], │ on index-opensearch.tf line 2, in resource "opensearch_ism_policy" "log_aws_default": │ 2: resource "opensearch_ism_policy" "log_aws_default" { │ ╵ ╷ │ Error: opensearch version 2.5.0 is older than 6.0.0 and is not supported, flavor: 0. │ │ with opensearch_index_template.log_aws_default[0], │ on index-opensearch.tf line 9, in resource "opensearch_index_template" "log_aws_default": │ 9: resource "opensearch_index_template" "log_aws_default" {
I want to create role with read access to all tenants.
resource "opensearch_role" "global_read_only" {
role_name = "global_read_only"
description = "Read only access to global tenants and indexes"
cluster_permissions = ["read", "get", "search"]
index_permissions {
index_patterns = ["*"]
allowed_actions = ["read", "get", "opensearch_dashboards_all_read", "search"]
}
tenant_permissions {
tenant_patterns = ["*"]
allowed_actions = ["read"]
}
}
After terraform apply I can see that read/write permission property is empty in OpenSearch console.
I am using AWS OpenSearch v 1.3.
Found in HEAD commit: cdaf5dde6d09b3313ee36aa19f53c1a0efc63a8f
CVE | Severity | Dependency | Type | Fixed in (github.com/hashicorp/go-hcLog-v1.5.0 version) | Remediation Available | |
---|---|---|---|---|---|---|
CVE-2022-41717 | 5.3 | golang.org/x/sys-v0.0.0-20220627191245-f75cf1eec38b | Transitive | N/A* | ❌ |
*For some transitive vulnerabilities, there is no version of direct dependency with a fix. Check the "Details" section below to see if there is a version of transitive dependency where vulnerability is fixed.
Library home page: https://proxy.golang.org/golang.org/x/sys/@v/v0.0.0-20220627191245-f75cf1eec38b.zip
Dependency Hierarchy:
Found in HEAD commit: cdaf5dde6d09b3313ee36aa19f53c1a0efc63a8f
Found in base branch: main
An attacker can cause excessive memory growth in a Go server accepting HTTP/2 requests. HTTP/2 server connections contain a cache of HTTP header keys sent by the client. While the total number of entries in this cache is capped, an attacker sending very large keys can cause the server to allocate approximately 64 MiB per open connection.
Publish Date: 2022-12-08
URL: CVE-2022-41717
Base Score Metrics:
Type: Upgrade version
Release Date: 2022-12-08
Fix Resolution: go1.19.4
Follow opensearch-project/.github#125 to baseline MAINTAINERS, CODEOWNERS, and external collaborator permissions.
Close this issue when:
If this repo's permissions was already baselined, please confirm the above when closing this issue.
Found in HEAD commit: cdaf5dde6d09b3313ee36aa19f53c1a0efc63a8f
CVE | Severity | Dependency | Type | Fixed in (github.com/aws/aws-sdk-go-v1.43.21 version) | Remediation Available | |
---|---|---|---|---|---|---|
CVE-2022-27664 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ | |
CVE-2022-30633 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ | |
CVE-2022-41721 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ | |
CVE-2022-32149 | 7.5 | golang.org/x/text-v0.3.7 | Transitive | N/A* | ❌ | |
CVE-2022-28131 | 7.5 | golang.org/x/net-v0.0.0-20220127200216-cd36cc0744dd | Transitive | N/A* | ❌ |
*For some transitive vulnerabilities, there is no version of direct dependency with a fix. Check the "Details" section below to see if there is a version of transitive dependency where vulnerability is fixed.
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: cdaf5dde6d09b3313ee36aa19f53c1a0efc63a8f
Found in base branch: main
In net/http in Go before 1.18.6 and 1.19.x before 1.19.1, attackers can cause a denial of service because an HTTP/2 connection can hang during closing if shutdown were preempted by a fatal error.
Publish Date: 2022-09-06
URL: CVE-2022-27664
Base Score Metrics:
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: cdaf5dde6d09b3313ee36aa19f53c1a0efc63a8f
Found in base branch: main
Uncontrolled recursion in Unmarshal in encoding/xml before Go 1.17.12 and Go 1.18.4 allows an attacker to cause a panic due to stack exhaustion via unmarshalling an XML document into a Go struct which has a nested field that uses the 'any' field tag.
Publish Date: 2022-08-10
URL: CVE-2022-30633
Base Score Metrics:
Type: Upgrade version
Origin: https://security-tracker.debian.org/tracker/CVE-2022-30633
Release Date: 2022-05-13
Fix Resolution: go1.17.12,go1.18.4
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: cdaf5dde6d09b3313ee36aa19f53c1a0efc63a8f
Found in base branch: main
A request smuggling attack is possible when using MaxBytesHandler. When using MaxBytesHandler, the body of an HTTP request is not fully consumed. When the server attempts to read HTTP2 frames from the connection, it will instead be reading the body of the HTTP request, which could be attacker-manipulated to represent arbitrary HTTP2 requests.
Publish Date: 2023-01-13
URL: CVE-2022-41721
Base Score Metrics:
Library home page: https://proxy.golang.org/golang.org/x/text/@v/v0.3.7.zip
Dependency Hierarchy:
Found in HEAD commit: cdaf5dde6d09b3313ee36aa19f53c1a0efc63a8f
Found in base branch: main
An attacker may cause a denial of service by crafting an Accept-Language header which ParseAcceptLanguage will take significant time to parse.
Publish Date: 2022-10-14
URL: CVE-2022-32149
Base Score Metrics:
Type: Upgrade version
Origin: https://www.cve.org/CVERecord?id=CVE-2022-32149
Release Date: 2022-10-14
Fix Resolution: v0.3.8
Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20220127200216-cd36cc0744dd.zip
Dependency Hierarchy:
Found in HEAD commit: cdaf5dde6d09b3313ee36aa19f53c1a0efc63a8f
Found in base branch: main
Uncontrolled recursion in Decoder.Skip in encoding/xml before Go 1.17.12 and Go 1.18.4 allows an attacker to cause a panic due to stack exhaustion via a deeply nested XML document.
Publish Date: 2022-08-10
URL: CVE-2022-28131
Base Score Metrics:
Type: Upgrade version
Origin: https://security-tracker.debian.org/tracker/CVE-2022-28131
Release Date: 2022-03-29
Fix Resolution: go1.17.12,go1.18.4
Persistent Terraform plan change detected with field config_id
opensearch_channel_configuration
resourceresource "opensearch_channel_configuration" "escalations" {
body = <<EOF
{
"config": {
"name": "name",
"description": "description",
"config_type": "slack",
"is_enabled": true,
"slack": {
"url": "${var.slack_escalations_channel_webhook}"
}
}
}
EOF
}
config_id
field (That was added by OpenSearch dashboard)No changes detected in plan
Operating system, version.
Add any other context about the problem.
Resource opensearch_cluster_settings
property action_auto_create_index
fails validation for legitimate string values.
opensearch_cluster_settings
action_auto_create_index
to -.aws_cold_catalog*,+*
String should pass validation and Terraform should apply
$ uname -a && terraform -version
Darwin bne-nb-ariel 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 arm Darwin
Terraform v1.5.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.62.0
+ provider registry.terraform.io/hashicorp/external v2.3.1
+ provider registry.terraform.io/hashicorp/local v2.4.0
+ provider registry.terraform.io/hashicorp/null v3.2.1
+ provider registry.terraform.io/hashicorp/random v3.5.1
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
+ provider registry.terraform.io/phillbaker/elasticsearch v2.0.4
Perhaps the variable should be either bool
or list(string)
instead?
I'm trying to map IAM roles to backend roles, but i'm getting an error:
Error: invalid character '<' looking for beginning of value
roles_mapping.tf:
resource "opensearch_roles_mapping" "mapper" {
role_name = "default"
backend_roles = ["arn:aws:iam::123456:role/myrole"]
description = "Mapping AWS IAM roles to ES role"
}
providers.tf:
provider "opensearch" {
url = "http://127.0.0.1:9200"
aws_assume_role_arn = "arn:aws:iam::123456:role/emizrahi-test-opensearch-master-dev"
healthcheck = false
}
versions.tf:
terraform {
required_version = "~> 1.3.6"
required_providers {
opensearch = {
source = "opensearch-project/opensearch"
version = "1.0.0"
}
}
}
Any ideas?
Terraform tries to set the number of replicas back to null, which will never work.
opensearch_index
type and set the only required property, name
number_of_replicas
number_of_replicas
to null
Either:
A) It's possible to set replica count to nothing, in which case Terraform does so
B) Number of replicas property should be mandatory
$ uname -a && terraform -version
Darwin bne-nb-ariel 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 arm Darwin
Terraform v1.3.9
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.7.0
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
Your version of Terraform is out of date! The latest version
is 1.5.2. You can update by downloading from https://www.terraform.io/downloads.html
Technically it's correct that replica count isn't required to create an index. But I'm no software philosopher.
Attempting to create an index template via terraform results in an [type=x_content_parse_exception]. The resource definition is the exact one taken from here: https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/index_template
Terraform file:
terraform {
required_providers {
opensearch = {
source = "opensearch-project/opensearch"
version = "2.0.0-beta.1"
}
}
}
resource "opensearch_index_template" "template_1" {
name = "template_1"
body = <<EOF
{
"template": "te*",
"settings": {
"number_of_shards": 1
},
"mappings": {
"type1": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z YYYY"
}
}
}
}
}
EOF
}
Nothing has been changed in the resource definition, it is the exact one from the provider page.
The index template is created.
AWS OpenSearch 2.7 domain
Terraform v1.4.5
opensearch-project/opensearch 2.0.0-beta.1
Existing release of the terraform provider does only support 1.x version of OpenSearch. Allow the same provider to support 2.x series of OpenSearch as well.
Support the provider to be compatible with 2.x series of OpenSearch.
Sample PR: #16
Have a separate PR with unit tests for features/changes that support 2.x series of OpenSearch while still supporting 1.x series.
opensearch_cluster_settings
unable to apply cluster_max_shards_per_node
Terraform is able to apply the changes, cluster query show the updated value.
$ uname -a && terraform -version
Darwin bne-nb-ariel 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 arm Darwin
Terraform v1.5.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.62.0
+ provider registry.terraform.io/hashicorp/external v2.3.1
+ provider registry.terraform.io/hashicorp/local v2.4.0
+ provider registry.terraform.io/hashicorp/null v3.2.1
+ provider registry.terraform.io/hashicorp/random v3.5.1
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
+ provider registry.terraform.io/phillbaker/elasticsearch v2.0.4
All using the same user as the Terraform provider configuration
Starting and using the provider locally in debug mode fails with this exception:
Error parsing %!q(PANIC=String method: called String on zero-value addrs.Provider) as a provider address: Invalid provider type: Provider source "opensearch-project/terraform-provider-opensearch" has a type with the prefix "terraform-provider-", which isn't valid. Although that prefix is often used in the names of version control repositories for Terraform providers, provider source strings should not include it.
Did you mean "opensearch-project/opensearch"?
The provider itself also prints an error message when started (./terraform-provider-opensearch -debuggable
):
{"@Level":"info","@message":"2023/09/15 08:46:41 [ERROR] Error parsing provider name "registry.terraform.io/opensearch-project/terraform-provider-opensearch": Invalid provider type: Provider source "opensearch-project/terraform-provider-opensearch" has a type with the prefix "terraform-provider-", which isn't valid. Although that prefix is often used in the names of version control repositories for Terraform providers, provider source strings should not include it.\n\nDid you mean "opensearch-project/opensearch"?","@timestamp":"2023-09-15T08:46:41.397308+02:00"}
Follow the steps described in the Readme
Using the provider locally in debug mode works without errors.
MacOS (x86) 13.4.1, Terraform v1.4.5
If aws_assume_role_arn is specified, but no profile is given,
the provider will assume that the 'default' profile will assume the given role arn.
This is not necessarily true, for instance if AWS credentials are specified via environment variables they should take the precedence and not force to use the 'default' profile.
provider "opensearch" {
url = "https://...."
aws_assume_role_arn = "arn:aws:iam::...:role/opensearch-role"
}
aws sts assume-role --role-arn arn:aws:iam::...:role/opensearch-build --role-session-name test
and make sure that the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN are set
6. TF_LOG=debug terraform apply:
you should get an error similar to this:
<Message>User: arn:aws:iam::...:user/some-user is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::...:role/opensearch-role</Message>
...
Error: NoCredentialProviders: no valid providers in chain. Deprecated.
No errors, since opensearch-build is allowed to assume opensearch-role
Macbook Pro - MacOS Ventura 13.2
Add any other context about the problem.
I want to provision and configure KNN index properties via terraform but you don't seem to support the args that are added via plugin here
Support configuring KNN index properties via terraform.
Use a script that runs after terraform to apply additional index configuration.
No
Found in HEAD commit: 4c08dd7a5e25b9e79d8de223f405277bd3f122b8
CVE | Severity | Dependency | Type | Fixed in (github.com/hashicorp/go-hclog-v1.2.0 version) | Remediation Available | |
---|---|---|---|---|---|---|
CVE-2022-41717 | 5.3 | golang.org/x/sys-v0.0.0-20220627191245-f75cf1eec38b | Transitive | N/A* | ❌ |
*For some transitive vulnerabilities, there is no version of direct dependency with a fix. Check the "Details" section below to see if there is a version of transitive dependency where vulnerability is fixed.
Library home page: https://proxy.golang.org/golang.org/x/sys/@v/v0.0.0-20220627191245-f75cf1eec38b.zip
Dependency Hierarchy:
Found in HEAD commit: 4c08dd7a5e25b9e79d8de223f405277bd3f122b8
Found in base branch: main
An attacker can cause excessive memory growth in a Go server accepting HTTP/2 requests. HTTP/2 server connections contain a cache of HTTP header keys sent by the client. While the total number of entries in this cache is capped, an attacker sending very large keys can cause the server to allocate approximately 64 MiB per open connection.
Publish Date: 2022-12-08
URL: CVE-2022-41717
Base Score Metrics:
Type: Upgrade version
Release Date: 2022-12-08
Fix Resolution: go1.19.4
Terraform attempts to delete cluster settings that can never be deleted.
opensearch_cluster_settings
with the minimal propertiesI'm no OpenSearch wizard but I'm pretty sure a cluster can't function without these.
So they were probably there before we even created the resource block and can probably never be deleted.
Given they're indestructible, perhaps the provider could throw a warning and simply remove it from state?
$ uname -a && terraform -version
Darwin bne-nb-ariel 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112 arm64 arm Darwin
Terraform v1.3.9
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.7.0
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
Your version of Terraform is out of date! The latest version
is 1.5.2. You can update by downloading from https://www.terraform.io/downloads.html
No.
Adding more than one opensearch_roles_mapping
per role will cause inconsistency between the terraform state and the actual mapping in OpenSearch.
OpenSearch's REST API provides an endpoint for role mapping using the role name as identifier:
_plugins/_security/api/rolesmapping/<role>
.
OpenSearch Documentation
Multiple opensearch_roles_mappings
for the same role will result in multiple calls to this REST API, overwriting or deleting existing role mappings.
Add role:
opensearch_roles_mappings
resources with the same role_name.Resource "opensearch_roles_mapping" "a" {
role_name = a_role
...
}
resource "opensearch_roles_mapping" "b" {
role_name = a_role
...
}
Remove one of these roles:
resource "opensearch_roles_mapping" "a" {
role_name = a_role
...
}
Is your feature request related to a problem? Please describe.
When I trying to use this provider with AWS hosted ES cluster the AWS signer auth support lacks, which makes hard to use this provider in such cases.
Describe the resource you would like to have implemented.
None
Describe the solution you'd like
Adding AWS auth support to provider
Describe alternatives you've considered
Currently I'm running local proxy (which implements requests signing)
Additional context
Add any other context or screenshots about the feature request here.
Can we create a terraform resource for Opensearch anomaly detection capabilities?
Thanks,
Tony
A clear and concise description of the bug.
I get the response:
"Your request: '/_opendistro' is not allowed."
when creating a role. It seems to be because the provider uses _opendistro
instead of _plugin
in the path.
This is using a Opensearch cluster created with the AWS Opensearch service, version 2.5.0
Steps to reproduce the behavior.
A clear and concise description of what you expected to happen.
Expected the role to be created.
Operating system, version.
If applicable, add screenshots to help explain your problem.
Add any other context about the problem.
With 2.0.0-beta.1
of the provider, we can't manage our existing Audit logs configuration anymore.
Terraform plan/apply
fails with:
╷
│ Error: audit config only available from OpenSearch >= 1.0, got version 2.7.0
│
│ with module.logs.opensearch_audit_config.audit[0],
│ on ../../../modules/opensearch/main.tf line 257, in resource "opensearch_audit_config" "audit":
│ 257: resource "opensearch_audit_config" "audit" {
│
╵
I guess this is related to this version check. AFAIK the /_plugins/_security/api/audit
API is supported in 1.x
and 2.x
of OpenSearch.
Create an opensearch_audit_config
resource using the 1.x
version of this provider and then try to apply again using 2.0.0-beta.1
(guess creating such a resource with 2.x will fail as well).
opensearch_audit_config
resources can be managed with terraform against OpenSearch 2.x clusters/domains.
AWS OpenSearch domain using 2.7.0
of OpenSearch
{
"name": "foo",
"cluster_name": "bar",
"cluster_uuid": "baz",
"version": {
"distribution": "opensearch",
"number": "2.7.0",
"build_type": "tar",
"build_hash": "unknown",
"build_date": "some-date",
"build_snapshot": false,
"lucene_version": "9.5.0",
"minimum_wire_compatibility_version": "7.10.0",
"minimum_index_compatibility_version": "7.0.0"
},
"tagline": "The OpenSearch Project: https://opensearch.org/"
}
Hi,
Trying to map roles to backend, after enabling FGAC on OpenSearch 2.3, using this block:
resource "opensearch_roles_mapping" "mapper" {
role_name = "logs_writer"
description = "Mapping AWS IAM roles to ES role"
backend_roles = [
"arn:aws:iam::123456789012:role/lambda-call-opensearch",
"arn:aws:iam::123456789012:role/run-containers",
]
}
provider version 2.0.0-beta.1
Getting:
│ Error: invalid character '<' looking for beginning of value
Did someone came across this issue and have a solution?
Opensearch destinations seem to be deprecated with 2.0 and have been superseded by notifications, see https://opensearch.org/docs/latest/observing-your-data/alerting/monitors/#questions-about-destinations and https://docs.aws.amazon.com/opensearch-service/latest/developerguide/alerting.html#alerting-notifications.
Using a opensearch_destination
ressource with this provider and a Opensearch domain 2.x fails with
│ Error: elastic: Error 405 (Method Not Allowed)
Support the notifications API in this provider: https://opensearch.org/docs/latest/observing-your-data/notifications/api/
Create notifications using direct API calls or using the Dashboards UI.
no
I've created an initial index test-logs-000001 which has is_write_index = true
property set and this index is managed by opensearch_index_template
resource and configured to rollover using ISM policy resource.
But as the rollover occurs, the index test-logs-000002, test-logs-000003...etc, the latest index becomes the write enabled index, and so when I subsequently run terraform plan and apply, it throws the error as below:
Error: elastic: Error 500 (Internal Server Error): alias [test-logs] has more than one write index [test-logs-000001,test-logs-00010] [type=illegal_state_exception]
is_write_index=true
resource "opensearch_ism_policy" "rollover_index_policy" {
policy_id = "Rollover-and-delete-indexes"
body = jsonencode(
{
"policy" : {
"description" : "Rollover index and delete it after 7 days",
"default_state" : "hot",
"states" : [
{
"name" : "hot",
"actions" : [
{
"retry" : {
"count" : 3,
"backoff" : "exponential",
"delay" : "1m"
},
"rollover" : {
"min_size" : "10gb"
}
}
],
"transitions" : [
{
"state_name" : "delete",
"conditions" : {
"min_index_age" : "7d"
}
}
]
},
{
"name" : "delete",
"actions" : [
{
"retry" : {
"count" : 3,
"backoff" : "exponential",
"delay" : "1m"
},
"delete" : {}
}
],
"transitions" : []
}
],
"ism_template" : [
{
"index_patterns" : [
"test-logs-*"
],
"priority" : "100"
}
]
}
}
)
}
resource "opensearch_index_template" "index_template" {
name = "test-logs"
body = jsonencode(
{
"index_patterns" : [
"test-logs-*",
],
"template" : "test-logs-*",
"settings" : {
"number_of_shards" : 3,
"number_of_replicas" : 1,
"plugins" : {
"index_state_management" : {
"rollover_alias" : "test-logs"
}
}
}
}
)
}
resource "opensearch_index" "index" {
name = "test-logs-000001"
aliases = jsonencode(
{
"test-logs" = {
"is_write_index" = true
}
}
)
depends_on = [opensearch_index_template.index_template]
}
$ uname -a
Linux PBL244 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
$ terraform version
Terraform v1.5.3
on linux_amd64
+ provider registry.terraform.io/opensearch-project/opensearch v1.0.0
terraform cannot import index template - not found
Create an index template in some way. In this case I create an index template with an older provider. Switch to the new provider and try to import, and it is not able to find the index.
The index template should import from the existing configuration by name
MacOS for the terraform client, Server cluster is AWS Opensearch 2.7
GET _cat/templates
- shows the templates
GET _index_template/*
- does not show templates
The cluster is running in compatibility mode, and the index template API may have changed between the provider that created the template and the one trying to import it. The console shows the index template though, so the new provider should be able to find it and import it.
Getting errors when trying to use aws_assume_role_arn.
Error: NoCredentialProviders: no valid providers in chain. Deprecated. For verbose messaging see aws.Config.CredentialsChainVerboseErrors
If I add directly the role of terraform(aws_profile) to the security_managers opensearch role, it works like a charm - but when trying to assume one of the roles that are already in there - I get that error message.
Try using the role assumption for any opensearch change and the above error shows up.
Role will get assumed and it's permissions applied when access opensearch.
AWS opensearch domain - Opensearch 2.5 cluster with fine grained access control applied.
AWS environment, opensearch domain version 2.5 with fine grained control enabled, trying to add role mapping vis the opensearch provider(version 1.0.0).
It works as long as I don't try to assume a role(using aws_profile), assuming a role throws the error mentioned above.
The relevant provider config in terraform looks like that(removing sensitive data):
provider "opensearch" {
url = "https://${aws_elasticsearch_domain.this.endpoint}"
aws_region = data.aws_region.current.name
sign_aws_requests = true
healthcheck = false
opensearch_version = "OpenSearch_2.5"
aws_assume_role_arn = "arn:aws:iam::xxxxxxxxxxxxxx:role/yyyyyyyyyyyyyyy"
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.