Is your feature request related to a problem? Please describe

Add upbound provider support when using eksctl bootstrap

Describe the solution you'd like

Easy way to try out both provider aws and upbound

Is your feature request related to a problem? Please describe

Currently we do not have tests. We should at least be able to validate compositions and XRDs. In the future we should be able to validate AWS resources as well.

Describe the solution you'd like

Ideally, I would like to be able to spin up a cluster for each PR instead of having a long running cluster to run tests against them. Testing against multiple release versions is preferred.

Describe alternatives you've considered

Additional context

Add any other context or screenshots about the feature request here.

we have networks.awsblueprints.io/v1alpha1 for aws-provider compositions of VPC:

https://github.com/aws-samples/crossplane-aws-blueprints/blob/9197d704ea8f55621aac4422bd166d6e3c44b583/compositions/aws-provider/vpc/vpc-xrd.yaml#L9-L18

and vpc.awsblueprints.io/v1beta1 for aws-jet-provider compositions of VPC:

https://github.com/aws-samples/crossplane-aws-blueprints/blob/9197d704ea8f55621aac4422bd166d6e3c44b583/compositions/terrajet-aws-provider/vpc/vpc-xrd.yaml#L10-L15

both refer to the same resource but capturing different spec representation of it. does it make sense to settle on something like the following for better naming conventions?

networks.awsblueprints.io/v1alpha1 for aws-provider and
networks.awsblueprints.io/v2alpha1 for aws-jet-provider

or even easier

networks.awsblueprints.io/v1 for aws-provider and
networks.awsblueprints.io/v2 for aws-jet-provider

after all these are samples, so versioning is less relevant. but naming inconsistencies could be confusing. thoughts?

Is your feature request related to a problem? Please describe

Have an example the AWS Containers Retail Sample with Crossplane.

The sample application depends on stateful services like MySQL, DynamoDB, Redis, and RabbitMQ.

This reference architecture should be useful for someone building a complex solution using crossplane to deploy the AWS managed resources for microservice based application.

Describe the solution you'd like

The solution should provide top level claim that contains the version of each microservices, the solutions should be able to deploy the application and corresponding services:

• Amazon Aurora
• Amazon DynamoDB
• Amazon MQ
• Amazon Elastic Cache

Additional context (Advanced Workshop)

This should be a building block for a more comprehensive/advanced workshop, that can include :

• gitops (flux or argocd)
• opa/gatekeeper
• secret manager
• deployment of the EKS cluster as part of claim
• show how to handle sub accounts for different environments (TEST, STAGE, PROD).

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

SHA c5cfbe0

What is your environment, configuration and the example used?

The EKS composition in compositions/aws-provider/eks used with the example from examples/aws-provider/composite-resources/eks/eks-claim.yaml, which has the following connection secret config:

  writeConnectionSecretToRef:
    name: xplane-eks-cluster

What did you do and What did you see instead?

After deployment, the secret xplane-eks-cluster is empty.

Additional Information

The problem is due to the `connectionDetails` on the cluster resource in `compositions/aws-provider/eks/eks-managed-node-group.yaml` which have the following definition for `kubeconfig`:


      connectionDetails:
        - name: value
          fromConnectionSecretKey: kubeconfig

The key value is not aligned with the XRD which is why the secret becomes empty:

  connectionSecretKeys:
    - kubeconfig

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

v0.1.0

What is your environment, configuration and the example used?

https://github.com/awslabs/crossplane-on-eks/tree/main/bootstrap/terraform

What did you do and What did you see instead?

Running destroy.sh to cleanup and running into this.

╷
│ Error: deleting EC2 Subnet (subnet-0d0c1d9a1782ecc16): DependencyViolation: The subnet 'subnet-0d0c1d9a1782ecc16' has dependencies and cannot be deleted.
│       status code: 400, request id: 15d530b6-9c74-43b5-928f-7854b6defd37
│ 
│ 
╵
╷
│ Error: deleting EC2 Subnet (subnet-0513e2114646f177b): DependencyViolation: The subnet 'subnet-0513e2114646f177b' has dependencies and cannot be deleted.
│       status code: 400, request id: 436038fd-1dc8-4927-89d9-ecdfe017dd25
│ 
│ 
╵
╷
│ Error: deleting EC2 Subnet (subnet-0e349deeef2ad5678): DependencyViolation: The subnet 'subnet-0e349deeef2ad5678' has dependencies and cannot be deleted.
│       status code: 400, request id: 1b9038f6-6f50-495f-bfea-9ec2fdb57c64
│ 
│ 
╵

This happening due to NLBs not getting cleaned up prior to VPC destroy.

Additional Information

No response

Is your feature request related to a problem? Please describe

MySQL on RDS is very common and we should add an example of that.
MySQL on K8s should be easy to implement.

Describe the solution you'd like

MySQL on RDS
MySQL on K8s

Describe alternatives you've considered

Additional context

Add any other context or screenshots about the feature request here.

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

main

What is your environment, configuration and the example used?

See upstream project issue - aws-ia/terraform-aws-eks-blueprints#1630

Modules should used pinned versions to remain on version v4 (latest is v4.32.1) or module should be refactored for v5

v5 - aws-ia/terraform-aws-eks-blueprints#1421

What did you do and What did you see instead?

module search and found unpinned version in your repo

Additional Information

No response

Is your feature request related to a problem? Please describe

We should add an example of using pod identities
https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html

Describe the solution you'd like

When using the terraform module or eksctl, I want to be able to specify which IAM identity strategy I want to use for my crossplane AWS provider pods.

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

latest

What is your environment, configuration and the example used?

latest

What did you do and What did you see instead?

cd bootstrap/terraform/
terraform init

Errors with

➜  terraform git:(main) ✗ terraform init                                                                                                                    arn:aws:eks:us-west-2:015299085168:cluster/cluster-1-dev

Initializing the backend...
Initializing modules...
Downloading git::https://github.com/aws-ia/terraform-aws-eks-blueprints.git for eks_blueprints_crossplane_addons.portworx.helm_addon...
Downloading git::https://github.com/aws-ia/terraform-aws-eks-blueprints.git for eks_blueprints_crossplane_addons.sysdig_agent.helm_addon...
╷
│ Error: Failed to expand subdir globs
│ 
│ subdir "modules/kubernetes-addons/helm-addon" not found
╵

╷
│ Error: Failed to expand subdir globs
│ 
│ subdir "modules/kubernetes-addons/helm-addon" not found

Additional Information

No response

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

v0.1.0

What is your environment, configuration and the example used?

Using terraform boostrap.
Default configuration.

What did you do and What did you see instead?

I'm trying to setup the stack to try it.
I run the following command:

terraform init
terraform apply
# The error occur a first time after this command
aws eks --region eu-west-1 update-kubeconfig --name crossplane-blueprints --alias crossplane-blueprints
terraform apply
# error occur again

Also, I tried to manually access the cluster, and it seems to be unreachable:

It's seems to be a problem in the cluster config right ?
Any ideas ?

Additional Information

No response

Options to deploy Helm Add-on provider to bootstrap cluster

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

v0.1.0

What is your environment, configuration and the example used?

https://github.com/awslabs/crossplane-on-eks/tree/main/bootstrap/terraform

What did you do and What did you see instead?

Ran terraform apply. Expected to see all pods up and running. The kube-prometheus-stack-grafana pod is in a crash loop backoff with this error in the logs.

grafana logger=provisioning t=2024-02-01T14:26:19.066649715Z level=error msg="Failed to provision data sources" error="Datasource provisioning error: datasource.yaml config is invalid. Only one datasource per organization can be marked as default"
grafana Error: ✗ Datasource provisioning error: datasource.yaml config is invalid. Only one datasource per organization can be marked as default

Additional Information

No response

Is your feature request related to a problem? Please describe

As of now for provisioning a aurora cluster we need to create aurora monitoring and rds proxy role outside of composition and pass the arn through patching .

Describe the solution you'd like

The 2 roles mentioned above can be created as part of the composition .

Describe alternatives you've considered

Creating the 2 roles out side and passing the arn of the role to be patched in the composition.

Additional context

NA

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

no releases around yet:)

What is your environment, configuration and the example used?

provider-kubernetes is not getting installed by terraform bootstrap.

At the same time, it is used by the following Compositions:

• https://github.com/aws-samples/crossplane-aws-blueprints/blob/main/compositions/aws-provider/example-application/example-application.yaml#L127
• https://github.com/aws-samples/crossplane-aws-blueprints/blob/main/compositions/aws-provider/irsa/irsa-exact.yaml#L99

What did you do and What did you see instead?

Add povider-kubernetes setup to terraform bootstrap similar to eksctl bootstrap:

• https://github.com/aws-samples/crossplane-aws-blueprints/blob/main/bootstrap/eksctl/crossplane/kubernetes-provider.yaml
• https://github.com/aws-samples/crossplane-aws-blueprints/blob/main/bootstrap/eksctl/crossplane/kubernetes-provider-config.yaml

Additional Information

No response

Is your feature request related to a problem? Please describe

resourceConfig was designed to contain commonalities for all resources such as:

• deletionPolicy
• name
• region
• tags
• providerConfigName

but the name resource is too generic and leads usage confusion.

Describe the solution you'd like

• rename resourceConfig to commonConfig to avoid the confusion
• deletionPolicy is being deprecated, replace with managementPolicies
• set a default providerConfigName to aws-provider-config and remove from claims

Is your feature request related to a problem? Please describe

The repository offers Compositions, and examples with the upstream Crossplane AWS provider, we would like to have similar compositions and examples using Upbound AWS provider.

Describe the solution you'd like

Go over the Compositions and Examples present and create parallel folders to have parity, any opportunities to improve them are welcome.

Describe alternatives you've considered

None

Additional context

Some of these work already started by adding boostrap assets and instructions to install the Upbound AWS provider using eksctl and terraform.

ARGO CD is not available to access it from DNS URL

Provide link to the example related to the quesiton

k8s-argocd-argocdar-b58ba5914a-1e2b2273bd87b3f1.elb.eu-central-1.amazonaws.com

Can anyone help?

Add composition for EKS Managed Node Groups

Add Kuberentes Add-ons to EKS Cluster

Is your feature request related to a problem? Please describe

eksctl and terraform bootstrap are amazing but there is also the possibility to bootstrap Crossplane with Crossplane.

Describe the solution you'd like

We can create bootstrap mechanism utilizing local(k3d/kind) Crossplane enabled bootstrap cluster to create EKS cluster including Crossplane installation and eventually hand over cluster control to itself.

@vfarcic has a nice video illustrating this approach https://www.youtube.com/watch?v=IlaYGgyg06o ( the samples there are Azure ones but the approach is fully applicable to AWS)

Is your feature request related to a problem? Please describe

With the introduction of composition functions in Crossplane v1.11, we should add example functions that maybe useful when it comes to composing these AWS resources.

Describe the solution you'd like

Some functions that maybe useful:

• Tagging AWS resources from both AWS providers without hard coding them
• Dynamic array generation. E.g. Create array elements based on certain condition / annotation.
• Generation of connection details in more sophisticated ways
• Use of Config in FunctionIO, possibly env specific configuration.

Describe alternatives you've considered

None

Additional context

Add any other context or screenshots about the feature request here.

Please describe your quesiton here

I would like to know, if Crossplane on EKS is something that is planned to be offered as official EKS Addon?

Provide link to the example related to the quesiton

Additional context

More

• Yes, I have checked the repo for existing issues before raising this question

Describe the solution you'd like

composition + example that create an AMP workspace

Is your feature request related to a problem? Please describe

With Crossplane v1.11, the new EnvironmentConfig CR is introduced. We should add examples that take advantage of this.

Describe the solution you'd like

• We could use this to show how you can configure IRSA with OIDC values from a configuration object.
• Pull KMS IDs from a configuration object
• Pull default password values for RDS instance.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context or screenshots about the feature request here.

Hello,

I am accepting a service name from my developer in the claim. This string is used to create the role. There are use cases where the string is longer than 64 char (string limit of a role name). I have an opa policy to invalidate such claims.

But if its possible to trim the string to 64 characters, I can use that to create the role.

Regards
Ashish

Please describe your question here

I am trying the emr-on-eks composition example from the crossplane-on-eks library.
My EMRContainer (for EMR job-run) resource status is synced but not ready.

The question is, why is my EMR job resource not being created?

Taking a look a the XEMRContainer resource, I see the following events, describing 2 kinds of different errors:

cannot use dry-run create to name composed resource: an empty namespace may not be set during creation
cannot apply the patch at index 9: status: no such field

Events:
Type Reason Age From Message

---

Normal SelectComposition 5m6s (x3 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io Successfully selected composition
Warning ComposeResources 5m6s defined/compositeresourcedefinition.apiextensions.crossplane.io composed resource "irsa-role-only": cannot apply the patch at index 1: status: no such field
Warning ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io composed resource "read-policy-entrypoint": cannot use dry-run create to name composed resource: an empty namespace may not be set during creation
Warning ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io composed resource "read-policy": cannot use dry-run create to name composed resource: an empty namespace may not be set during creation
Warning ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io composed resource "write-policy": cannot use dry-run create to name composed resource: an empty namespace may not be set during creation
Warning ComposeResources 5m6s defined/compositeresourcedefinition.apiextensions.crossplane.io composed resource "job-run": cannot apply the patch at index 9: status: no such field
Normal ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io Composed resource "irsa-role-only" is not yet ready
Normal ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io Composed resource "read-policy-entrypoint" is not yet ready
Normal ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io Composed resource "read-policy" is not yet ready
Normal ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io Composed resource "write-policy" is not yet ready
Normal ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io Composed resource "s3-bucket" is not yet ready
Normal ComposeResources 5m6s (x2 over 5m6s) defined/compositeresourcedefinition.apiextensions.crossplane.io Composed resource "job-run" is not yet ready
Warning ComposeResources 5m6s defined/compositeresourcedefinition.apiextensions.crossplane.io composed resource "irsa-role-only": cannot apply the patch at index 1: status.accountId: no such field
Warning ComposeResources 5m6s defined/compositeresourcedefinition.apiextensions.crossplane.io composed resource "job-run": cannot apply the patch at index 9: status.roleArn: no such field

I created the EMRContainer for virtual cluster successfully, but the job-run resource fail with the above errors.

This is the XR for the jobs creation:

apiVersion: awsblueprints.io/v1alpha1
kind: EMRContainer
metadata:
  name: test-job-run
  namespace: dev
spec:
  compositionSelector:
    matchLabels:
      awsblueprints.io/environment: dev
      awsblueprints.io/type: job-run
  resourceConfig:
    providerConfigName: aws-provider
    region: eu-central-1
  eksOIDC: oidc.eks.eu-central-1.amazonaws.com/id/XXXXXXXXXXXXXX # Changed with suitable OIDC
  jobParams:
    sparkEntryPoint: s3://us-west-2.elasticmapreduce/emr-containers/samples/wordcount/scripts/wordcount.py
    sparkSubmitParameters: "--conf spark.executor.instances=2 --conf spark.executor.memory=1G --conf spark.executor.cores=1 --conf spark.driver.cores=1"
    virtualClusterId: "XXXXXXXXXXXXX" # Changed with suitable virtual cluster id

Provide link to the example related to the question

https://github.com/awslabs/crossplane-on-eks/blob/main/examples/aws-provider/composite-resources/emr-on-eks/job-run.yaml

Additional context

I installed crossplane latest with AWS provider package xpkg.upbound.io/crossplane-contrib/provider-aws:v0.37.1.

Is that possible that the latests commits in the compositions are affecting the stability of the examples?
9719c58

More

• Yes, I have checked the repo for existing issues before raising this question

Is your feature request related to a problem? Please describe

A few users are looking for an example to implement a REST-API using API-Gateway and Lambda with a Lambda authorizer

Describe the solution you'd like

Using upbound provider to create an example

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

latest

What is your environment, configuration and the example used?

I followed the doc doc/vault-integration.md to install and setup vault.

What did you do and What did you see instead?

Following the composition example for multitenant the secret is written to vault in the correct location for application1

But the secret is not created in namespace crossplane-system as stated in the docs.

kubectl -n crossplane-system get secret `kubectl get xobjectstorage -o json | jq -r '.items[0].metadata.uid'` -o go-template='{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}'
# example output
# bucket-name: standard-object-storage-qlgvz-hz2dn
# region: us-west-2

Additional Information

No response

Is your feature request related to a problem? Please describe

The terraform crossplane module is not longer offerred by the eks terraform blueprints git repository as part of the v5 generation

Describe the solution you'd like

The plan is to move the module crossplane to crossplane-on-eks git repo and refactor to use the new eks-blueprints-addon as suggested in aws-ia/terraform-aws-eks-blueprints-addons#67

Describe alternatives you've considered

No other alternatives at this moment

Additional context

This is the EKS Terraform Blueprints UPGRADE V5

Hello,

I have upgrade my crossplane deployment to xpkg.upbound.io/upbound/provider-aws-eks:v1.0.0. I have started to use Podidentityassociation.

Crossplane was successfully able to create the association and synced, but the ready state is false 😕 Below is the error of the managed resource. I dont have any previous managed resource / already existing association with an association id a-stubassocid123456

async create failed: resource creation call returned error diags: creating Amazon EKS (Elastic Kubernetes) Pod Identity Association ("a-stubassocid123456"): operation error EKS: CreatePodIdentityAssociation, https response error StatusCode: 409, RequestID: f9cb61ba-2e55-4c0e-a8f7-962bfd8302bc, ResourceInUseException: Association already exists: a-dt1bedfjc72dlwp5n: operation error EKS: CreatePodIdentityAssociation, https response error StatusCode: 409, RequestID: f9cb61ba-2e55-4c0e-a8f7-962bfd8302bc, ResourceInUseException: Association already exists: a-dt1bedfjc72dlwp5n

Is your feature request related to a problem? Please describe

The EKS cluster composition in compositions/aws-provider/eks require manual specification of subnet Ids. In Kubernetes, labels are typically used to reference resources, i.e. labelling subnets and using the resourcesVpcConfig.subnetIdSelector.matchLabels feature of the cluster.eks.crossplane.io resource would be more convenient and less error prone. Prior art for this can be found in https://github.com/upbound/platform-ref-aws

Describe the solution you'd like

A working example can be found here: https://github.com/MichaelVL/crossplane-aws-blueprints/tree/feature/subnet-label-selector

This working example implements the following

• Adds a new parameter networkId to the xvpcsubnets composition, which is propated to the subnet label network.awsblueprints.io/network-id. The idea behind NOT using the VPC name is that this is a logical network name and may consist of more than a VPC. The label use dash-case as is common for Kubernetes labels.
• Similarly adds a networkId to the xamazoneks composition and which is used for subnet selection in the cluster resourcesVpcConfig settings and similarly for the node group.
• Updates example claims with the networkId parameter.

I will be happy to make a PR from this working example.

Describe alternatives you've considered

None

Additional context

The 'working example' was heavily inspired by https://github.com/upbound/platform-ref-aws

When I create an irsa, it only creates a service account and role that works in the crossplane.io/claim-namespace. However I need another namespace to access the resource. How can I create an irsa in a different namespace that can access the resource (eg a bucket) that was created in the current namespace?

Is your feature request related to a problem? Please describe

ArgoCD + deploy clusters with Crossplane is a popular choice of deployment model. We should make an example of it with EKS addons and other needed things.

Describe the solution you'd like

1. Use ArgoCD annotation based tracking to avoid unwanted pruning.
2. Use Crossplane + packages (possibly custom) to provision EKS with popular addons installed.

We are not enabling debugging for AWS providers when deploying via terraform. This is what we have for the terraform installation:

https://github.com/aws-samples/crossplane-aws-blueprints/blob/9197d704ea8f55621aac4422bd166d6e3c44b583/bootstrap/terraform/main.tf#L186-L197

vs what we have on the AWS provider:

https://github.com/aws-samples/crossplane-aws-blueprints/blob/9197d704ea8f55621aac4422bd166d6e3c44b583/bootstrap/eksctl/crossplane/aws-provider.yaml#L20-L24

Also, do we have the possibility to enable debugging for the jet provider as well. if so, we need to do it:

https://github.com/aws-samples/crossplane-aws-blueprints/blob/9197d704ea8f55621aac4422bd166d6e3c44b583/bootstrap/eksctl/crossplane/jet-aws-provider.yaml#L20-L22

Is your feature request related to a problem? Please describe

We have added example for Cluster Autoscaler, and I think it's a good idea to add an example for Karpenter.

Describe the solution you'd like

Add compositions and examples for creating EKS clusters with Karpenter. This must be done in the upbound provider because the community provider does not support necessary cloudwatch resources.

Many of our customers use permissions boundary when creating roles. Admin role with no permissions boundary is likely not going to be acceptable for a lot of our customers. We should add support for it during bootstrapping.

Is your feature request related to a problem? Please describe

Have an example on how to configure argocd for production with crossplane

Describe the solution you'd like

Add argocd to the eksctl and terraform bootstraps

Describe alternatives you've considered

ArgoCD resource tracking needs to be done with annotations
Some special health checks need to be configure for claim resources

Additional context

More info in crossplane docs https://docs.crossplane.io/v1.10/guides/argo-cd-crossplane/#configuring-argo-cd-with-crossplane

Is your feature request related to a problem?

It would be an awesome feature if we could also support RDS Clusters with serverless v2 engine-mode. This would be a great benefit for everyone who doesn't have steady load on its database and just has to ramp up when load peaks occure.

Describe the solution you'd like

When I'm correct all we need to do in order to support this instance type is introducing serverlessV2ScalingConfiguration parameter to the DBCluster ResourceDefinitions and Compositions

Additional context

Add any other context or screenshots about the feature request here.
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.create-cluster.html

Please describe your quesiton here

I'm using Terraform resource "kubectl_manifest" to create a Crossplane SecurityGroup for an existing TF-created security group:

apiVersion: ec2.aws.upbound.io/v1beta1
kind: SecurityGroup
metadata:
  name: postgres
  annotations:
    crossplane.io/external-name: ${postgres-sg-id}
spec:
  managementPolicies: ["Observe"]

This is later referenced by a (non-Terraform) k8s manifest that creates an RDS Instance.
(This appears to be a good way for a Crossplane manifest to obtain the ID of an SG created by TF).

However, managementPolicies: ["Observe"] requires the ControllerConfig to have a --enable-management-policies argument:

spec:
  args:
    - "--enable-management-policies"

The blueprint doesn't appear to provide a way to specify this.
I have currently hacked it by using resource "null_resource" "k8s_patcher" to run kubectl patch.

It would be good if the blueprint provided a way to specify additional args.

In the meantime, any suggestions for better workarounds?

More

• Yes, I have checked the repo for existing issues before raising this question

Please describe your quesiton here

Hi,

I saw the same error as mentioned in:

https://github.com/awslabs/crossplane-on-eks/blob/main/doc/debugging.md#composition

I noticed the document mentioned For this particular issue, we need to specify the namespace for the IRSA resource.

Could you please elaborate how can we specify the namespace in a Composite and how did you fix this?

Should this be done through a patch/parameter, something like:

Fix 1:

    - type: FromCompositeFieldPath
      fromFieldPath: spec.parameters.managedResourceNamespace
      toFieldPath: metadata.namespace

Or did you fix it by

Fix 2: Adding metadata.namespace in the base of Composition directly.

I tried both and it seems none of them worked for me.

Fix 1 is giving me a new error:

  Warning  ComposeResources         20s (x6 over 77s)  defined/compositeresourcedefinition.apiextensions.crossplane.io  cannot compose resources: cannot associate composed resources with Composition resource templates: cannot get composed resource: an empty namespace may not be set when a resource name is provided

Fix 2 is not making any difference as I suspect the namespace specified in base is discarded by crossplane controller.

Provide link to the example related to the quesiton

Additional context

More

• Yes, I have checked the repo for existing issues before raising this question

Is your feature request related to a problem? Please describe

There is no EKS composition for the upbound AWS provider, only for the contrib one

Describe the solution you'd like

Create an EKS composition for the upbound provider

Describe alternatives you've considered

Each of us can create our own composition, but it would be beneficial for us all to have an "official" one that adheres to AWS best practices and will be actively maintained

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

latest

What is your environment, configuration and the example used?

the out of the box example for terraform

What did you do and What did you see instead?

normal deployment with terraform

git clone https://github.com/awslabs/crossplane-on-eks
cd bootstrap/terraform
terraform init
terraform apply -auto-approve

Errored out with

│ Error: aws-controller-config failed to create kubernetes rest client for update of resource: Get "https://xxx.gr7.us-east-1.eks.amazonaws.com/api?timeout=32s": dial tcp: lookup xxx.gr7.us-east-1.eks.amazonaws.com on [xxx:a601:a604:500::1]:53: no such host
│ 
│   with module.eks_blueprints_kubernetes_addons.module.crossplane[0].kubectl_manifest.aws_controller_config[0],
│   on .terraform/modules/eks_blueprints_kubernetes_addons/modules/kubernetes-addons/crossplane/main.tf line 20, in resource "kubectl_manifest" "aws_controller_config":
│   20: resource "kubectl_manifest" "aws_controller_config" {

Additional Information

No response

Is your feature request related to a problem? Please describe

With 1.14 Release of Crossplane Release and the opensource P&T function we should look to convert the Crossplane on EKS Compositions to use the function. There is a conversion CLI here https://github.com/crossplane-contrib/crossplane-migrator that can be used to help automate this process.

Describe the solution you'd like

I'd like to see the conversion to P&T function be done and see compositions usable with composition function pipelines.

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

sha:1ab8685

What is your environment, configuration and the example used?

Crossplane 1.9.1
crossplane/provider-aws:v0.31.0

What did you do and What did you see instead?

Tried to create an S3 bucket in eu-west-2:

apiVersion: awsblueprints.io/v1alpha1
kind: ObjectStorage
metadata:
  name: my-backup
  namespace: default
spec:
  compositionSelector:
    matchLabels:
      awsblueprints.io/provider: aws
      awsblueprints.io/environment: dev
      s3.awsblueprints.io/configuration: standard
  writeConnectionSecretToRef:
    name: bucket-info
  resourceConfig:
    providerConfigName: aws-provider
    region: eu-west-2

The bucket was created in us-west-2 instead.

Additional Information

Maybe this line is related: https://github.com/aws-samples/crossplane-aws-blueprints/blob/main/compositions/aws-provider/s3/general-purpose.yaml#L46

Welcome to AWS Blueprints for Crossplane!

• Yes, I've searched similar issues on GitHub and didn't find any.

AWS Blueprints for Crossplane Release version

v0.1.0

What is your environment, configuration and the example used?

https://github.com/awslabs/crossplane-on-eks/tree/main/bootstrap/terraform

What did you do and What did you see instead?

The gavinbunney/kubectl provider is no longer actively maintained and is known to be very unstable and unusable at this point. I did a minor change in main.tf and reapplied and saw these errors which are typical symptoms of the unstable state of the provider. Across EKS Blueprints we are discontinuing the use of the kubectl_manifest and recommending folks to use the Hashicorp terraform-provider-kubernetes or package up the yaml files as inline helm charts.

Error: cluster failed to fetch resource from kubernetes: the server could not find the requested resource
│ 
│   with kubectl_manifest.environmentconfig,
│   on main.tf line 196, in resource "kubectl_manifest" "environmentconfig":
│  196: resource "kubectl_manifest" "environmentconfig" {
│ 
╵
╷
│ Error: upbound-aws-controller-config failed to fetch resource from kubernetes: the server could not find the requested resource
│ 
│   with kubectl_manifest.upbound_aws_controller_config[0],
│   on main.tf line 289, in resource "kubectl_manifest" "upbound_aws_controller_config":
│  289: resource "kubectl_manifest" "upbound_aws_controller_config" {
│ 
╵
╷
│ Error: kubernetes-controller-config failed to fetch resource from kubernetes: the server could not find the requested resource
│ 
│   with kubectl_manifest.kubernetes_controller_config[0],
│   on main.tf line 418, in resource "kubectl_manifest" "kubernetes_controller_config":
│  418: resource "kubectl_manifest" "kubernetes_controller_config" {
│ 
╵
╷
│ Error: helm-controller-config failed to fetch resource from kubernetes: the server could not find the requested resource
│ 
│   with kubectl_manifest.helm_controller_config[0],
│   on main.tf line 482, in resource "kubectl_manifest" "helm_controller_config":
│  482: resource "kubectl_manifest" "helm_controller_config" {

Additional Information

No response

Is your feature request related to a problem? Please describe

Review which Terraform locals should be exposed as variables

Please describe your quesiton here

I have copied and pasted the examples used here to create an S3 bucket using a composite resource definition and I seeing an error and the buckets are not being create, though I have not mad any changes to the examples specified in this repository:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xobjectstorages.awsblueprints.io
spec:
  claimNames:
    kind: ObjectStorage
    plural: objectstorages
  group: awsblueprints.io
  names:
    kind: XObjectStorage
    plural: xobjectstorages
  connectionSecretKeys:
    - region
    - bucket-name
    - s3-put-policy
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          properties:
            spec:
              description: ObjectStorageSpec defines the desired state of ObjectStorage
              properties:
                resourceConfig:
                  description:
                    ResourceConfig defines general properties of this AWS
                    resource.
                  properties:
                    deletionPolicy:
                      description: Defaults to Delete
                      enum:
                        - Delete
                        - Orphan
                      type: string
                    name:
                      description:
                        Set the name of this resource in AWS to the value
                        provided by this field.
                      type: string
                    providerConfigName:
                      type: string
                    region:
                      type: string
                    tags:
                      items:
                        properties:
                          key:
                            type: string
                          value:
                            type: string
                        required:
                          - key
                          - value
                        type: object
                      type: array
                  required:
                    - providerConfigName
                    - region
                    - tags
                  type: object
              required:
                - resourceConfig
              type: object
            status:
              description: ObjectStorageStatus defines the observed state of ObjectStorage
              properties:
                bucketName:
                  type: string
                bucketArn:
                  type: string
              type: object
          type: object
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: s3bucket.awsblueprints.io
  labels:
    awsblueprints.io/provider: aws
    awsblueprints.io/environment: dev
    s3.awsblueprints.io/configuration: standard
spec:
  writeConnectionSecretsToNamespace: crossplane-system
  compositeTypeRef:
    apiVersion: awsblueprints.io/v1alpha1
    kind: XObjectStorage
  patchSets:
    - name: common-fields
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.providerConfigName
          toFieldPath: spec.providerConfigRef.name
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.deletionPolicy
          toFieldPath: spec.deletionPolicy
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.region
          toFieldPath: spec.forProvider.region
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.name
          toFieldPath: metadata.annotations[crossplane.io/external-name]
  resources:
    - name: s3-bucket
      connectionDetails:
        - name: bucket-name
          fromConnectionSecretKey: endpoint
        - name: region
          fromConnectionSecretKey: region
      base:
        apiVersion: s3.aws.crossplane.io/v1beta1
        kind: Bucket
        spec:
          deletionPolicy: Delete
          forProvider:
            acl: private
            publicAccessBlockConfiguration:
              blockPublicAcls: true
              blockPublicPolicy: true
              ignorePublicAcls: true
              restrictPublicBuckets: true
            serverSideEncryptionConfiguration:
              rules:
                - applyServerSideEncryptionByDefault:
                    sseAlgorithm: AES256
      patches:
        - type: PatchSet
          patchSetName: common-fields
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.tags
          toFieldPath: spec.forProvider.tagging.tagSet
          policy:
            mergeOptions:
              appendSlice: true
              keepMapValues: true
        - type: FromCompositeFieldPath
          fromFieldPath: spec.resourceConfig.region
          toFieldPath: spec.forProvider.locationConstraint
        - fromFieldPath: spec.writeConnectionSecretToRef.namespace
          toFieldPath: spec.writeConnectionSecretToRef.namespace
        - type: ToCompositeFieldPath
          fromFieldPath: metadata.annotations[crossplane.io/external-name]
          toFieldPath: status.bucketName
        - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.arn
          toFieldPath: status.bucketArn
        - fromFieldPath: metadata.uid
          toFieldPath: spec.writeConnectionSecretToRef.name
          transforms:
            - type: string
              string:
                fmt: "%s-bucket"
---
apiVersion: awsblueprints.io/v1alpha1
kind: ObjectStorage
metadata:
  name: standard-object-storage
  namespace: default
spec:
  compositionSelector:
    matchLabels:
      awsblueprints.io/provider: aws
      awsblueprints.io/environment: dev
      s3.awsblueprints.io/configuration: standard
  writeConnectionSecretToRef:
    name: bucket-info
  resourceConfig:
    providerConfigName: aws-provider-config
    region: eu-central-1
    tags:
      - key: env
        value: test
      - key: anotherKey
        value: anotherValue

Crossplane configs:

kind: ControllerConfig
metadata:
name: aws-config
annotations:
    eks.amazonaws.com/role-arn: "$ROLE_ARN"
spec:
podSecurityContext:
    fsGroup: 2000

apiVersion: aws.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
name: aws-provider
spec:
credentials:
    source: InjectedIdentity

crossplane was installed with helm (repository: https://charts.crossplane.io/stable, version: 1.10.1) with this version:
"{xpkg.upbound.io/crossplane-contrib/provider-aws:v0.33.0}"

Here is the error I get:

• Yes, I have checked the repo for existing issues before raising this question