Multiple claims on a single composite resource about crossplane HOT 9 CLOSED

This composition functions pipeline model is definitely the coming of age of Crossplane - as we've covered it's very powerful once more functions land and expand the capabilities of what you can do in the pipelines.

I guess currently it's not possible, and that's fine - For now I'll opt to simplify my stack rather than running multiple stages of environment configs all loosely held together by naming convention and string.

I'm looking forward to what's coming next with Crossplane and if we end up with the capabilities to evaluate other CompositeResources, or ManagedResources down to the field level as if they were native to the composition it would open up some fantastic options that as far as I can tell, would be nigh on impossible in any other IAC tooling.

Thanks for taking the time to reply and help me understand, as you can imagine this has been a whistle stop tour of Crossplane for me this week!

from crossplane.

The current implementation is only retrieving EnvironmentConfigs - https://github.com/crossplane-contrib/function-environment-configs/blob/main/fn.go#L257

I know @phisco wants to expand it to other/generic resources but I don't know if he has any timeline in mind.

from crossplane.

Claim-Composite is a 1-to-1 relationship today. What is the scenario that you are thinking about to make multiple claims point at a single composite?

from crossplane.

As a side note, if we are allowing a second Claim to "take over" an existing Composite from it's Claim, that's a bug that we should fix. It's effectively orphaning the claim.

from crossplane.

Thanks for the reply:

Firstly the scenario is for creating some infrastructure in an EKS cluster that multiple, unpredictable, namespaces can get some connection secrets.

If we take the simplest and ugliest model:

• S3 bucket
• Policy
• UserPolicyAttachment
• User
• AccessKey

And you want to output those Access keys into various namespaces you have a couple options:

• Create the XR (no need for claims) and use one of many kubernetes operators to reflect this outputted connection secret to namespaces.
• Create an XR in the Crossplane namespace, and use multiple claims (one per namespace) to create connection secrets in the local namespace they want to be used in.

Secondly, I haven't found this to be the case - if you create multiple Claim objects with kubectl at the same time then one of them will secure the claim ownership of the XR and the others fail - as expected seemingly.

This was mostly musing, I've seen the thread about mapping the ideas to PVC > PVs in the storage world but given multiple claims can point to the same PV that wasn't as ideal as I'd hoped.

Since thinking about the problem I'm potentially just creating way more work for myself than is worth it and I'm going to just drop the cluster wide resource idea - it's additionally painful mapping cluster wide resources with PodIdentityAssociation objects that are themselves targetting namespaces and service accounts, which can't be known ahead of time!

from crossplane.

I think you might be able to do something like this using EnvironmentConfig - create your "infra" composite so that it creates an EnvironmentConfig resource and publishes the name/namespace of it's Connection Secret to the EnvironmentConfig. Then in your "consumer" composite(s) select that EnvironmentConfig by name or labels, retrieve the ConnectionSecret information using an Observe-only provider-kubernetes Object, and create a new Secret in the target namespace using a second provider-kubernetes Object.

Another option would be to use a Composition Function with the new "extra resources" functionality - the consuming Composite would need to know the name of the infra Composite, but then the Composition Function could request that Crossplane retrieve the infra composite and it's connection secret and then publish that into a Secret using the Object.

from crossplane.

These are some interesting ideas, and possibly worth pursuing - the first one I can effectively do with kyverno using the generation functionality and sync the secrets I guess - but it's nice to know a native Crossplane solution.

The second option is interesting, is there a function already that operates like a Terraform Data source object to do cross composition lookups, because as you say just reaching into a composite resource and grabbing some fields to use in a new composition would be extremely powerful and be somewhat of a general purpose solution for any of these problems.

I assume you're referencing this design doc - https://github.com/crossplane/crossplane/blob/master/design/design-doc-composition-functions-extra-resources.md though I have no idea if it's been implemented.

For example - I could populate the EKS cluster with an observe only EKS cluster resource (as part of a composition) that I could reference fields from the status - AWS region, VPC ID etc.

Currently I can only reference a cluster like that in MRs that support a clusterNameRef type field (see PodIdentityAssociation), and only then it's getting just the name (raised an issue here).

I think the pattern of being able to reach into existing composite resources, and access the fields on the underlying managed resources would be extremely useful.

I could for example:

• Provision an S3 bucket at the cluster level as a cluster admin.
• Via a second composite resource, carve out a prefix from that bucket by name, create the associated IAM policy/role, and attach a PodIdentityAssociation resource so pods can use that prefix scoped access.

Currently it's proving really difficult to slice and dice up a bucket so namespace/service account scoped PodIdentityAssociation objects can dish out those permissions to pods.

So much so up until you replied I was just going to suck up s3 buckets per claim (per application per namespace) simply because cohabitating was so difficult.

from crossplane.

Sometimes it's definitely easier to use unique resources rather than create/maintain the complexity of shared resources, especially if cost is not an issue.

function-environment-config uses the extra resources functionality to request EnvironmentConfig resources and create the environment to pass to function-patch-and-transform or any other function that wants the environment. The same concepts can be used more dynamically/generically for what you describe.

from crossplane.

That's true, though I'm more concerned with AWS quotas (albeit a soft limit) on buckets (100 per account) - and I'm building a SaaS product which might end up with (hopefully) many more than 100 accounts.

I was already using the EnvironmentConfig to pass through some things between compositions, to the function-go-templating function - namely the AWS_REGION and EKS_CLUSTER_NAME which were handy in other compositions.

Just so I'm clarifying, the second option above talked about "extra resources" functionality. Is that part of the function-environment-config function, or is that yet to be implemented?

The example in the design doc here looks promising but it seemed to have been descoped for now?

Likewise this syntax is promising.

---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: nop.sqlinstances.example.org
spec:
  compositeTypeRef:
    apiVersion: example.org/v1alpha1
    kind: XSQLInstance
  mode: Pipeline
  pipeline:
  - name: envconfigs
    functionRef:
      name: function-environment-configs
    input:
      apiVersion: extra.fn.crossplane.io/v1beta1
      kind: ExtraResourceSelectors
      extraResourceSelectors:
        environmentConfigs:
        - ref:
            name: example-environment-1
        - type: Selector
          selector:
            mode: Multiple
            sortByFieldPath: data.priority
            matchLabels:
              - type: FromCompositeFieldPath
                key: stage
                valueFromFieldPath: metadata.labels[stage]
  - name: patch-and-transform
    functionRef:
      name: function-patch-and-transform
    input:
      apiVersion: pt.fn.crossplane.io/v1beta1
      kind: Resources
      resources:
        - name: nop
          base:
            apiVersion: nop.crossplane.io/v1alpha1
            kind: NopResource
            spec:
              forProvider:
                conditionAfter:
                  - conditionType: Ready
                    conditionStatus: "False"
                    time: 0s
                  - conditionType: Ready
                    conditionStatus: "True"
                    time: 1s
          patches:
            - type: FromEnvironmentFieldPath
              fromFieldPath: complex.c.f
              toFieldPath: metadata.annotations[valueFromEnv]

I appreciate this is timely and cutting edge so the implementations and documentation is new and rough, but I am trying to get a grip with what is and isn't available in this respect - and by this I explicitly mean the capacity to lookup external resources and then interact with them in the next pipeline step as if they were part of the composition.

from crossplane.