Docs or guidance on writing additional third party fan out integrations about chainloop HOT 4 CLOSED

I love "The short answer is no" and then the longer answer being all the details that would be needed, along with all of the context 🥇

I'll definitely take a look if/when I have time, and will track the issue mentioned.

Thanks for the detailed response.

from chainloop.

Hi @garethr!

The short answer is no :( we don't have documentation yet and in fact there is probably some work to do first to make sure the abstractions are properly set or to create an actual plugins architecture.

I can try to in the meantime explain what I believe would be the steps involved

1 - Implement the actual integration code

A integration needs to implement this interface.

and you can see the only integration we have right now (Dependency-Track for SBOMs) that implements this interface here

NOTE: OCI registries although they are depicted in our diagrams as fan-out integrations, it's implementation doesn't follow this approach for historical reasons, but it will fix with this #28

2 - Registration logic

2.1 - Add use-case business logic

This means that an user can register their instance of the integration in their control-plane org. Basically this step

To do that you'd need to add an useCase to register the new integration in the biz layer. Similar to

that code will validate and store any credentials in the secrets storage, i.e dependency-track API key, and store an entry in the DB.

2.2 - Add API endpoint

Next, you'll need to add an API endpoint to add your new integration.

Similar to this one

3 - Enable attachment to workflows

At this point you'll have a mechanism for your users to add this new integration to their organization, so they'll be able to get smth like

$ chainloop integration ls
┌──────────────────────────────────────┬──────────────────┬──────────────────────────────────────────────┬─────────────────────┐
│ ID                                   │ KIND             │ CONFIG                                       │ CREATED AT          │
├──────────────────────────────────────┼──────────────────┼──────────────────────────────────────────────┼─────────────────────┤
│ c48f3041-5745-4773-aed9-ccf2f288b2e4 │ Dependency-Track │ host: https://dependency-track.chainloop.dev │ 21 Dec 22 16:29 UTC │
│                                      │                  │ allowAutoCreate: true                        │                     │
└──────────────────────────────────────┴──────────────────┴──────────────────────────────────────────────┴─────────────────────┘

Next, you'll need to implement how to allow your users to attach this new integration to their workflow, so it gets executed once an attestation is received.

to do that

3.1 - Extend API with custom configuration

If the attachment command requires additional, specific configurations, you'll need to extend this oneof and it's handling

3.2 - Handle attach to workflow logic

4 - Implement the actual firing mechanism

Now our new integration is ready to 1) be registered by any organization and 2) be attached to as many workflows as needed.

Next, we need to implement the actual actuation of the integration code you implemented in step 1.

And this is by far the part that needs more work. I have pending adding an issue about adding an event bus (nats.io or similar) to handle persistence, retries, etc.

but for now the code is executed in the service implementation when the attestation is received, like for example

I know that this is a lot to digest and it's current architecture is not great. As I mentioned before, it might make sense to try to encapsulate most of these changes as part of a new registration/attachment interface, so the custom code related to adding a new integration will be mostly encapsulated in the first step.

I am going to take a look at that as part of #28 but if you have any thoughts, want to chat or help please let me know!!

Thank you!

from chainloop.

Sounds good! and yes, I error on the verbosity side as long as a TLDR is in place haha :)

For the record, and to elaborate on my previous statement...

... there is probably some work to do first to make sure the abstractions are properly set or to create an actual plugins architecture.

These are some of the additional work streams around third-party integrations that I have on top of my priority list.

• #38
• #39

In any case @garethr, if you have in mind any specific integration you'd like to explore or have added to Chainloop, feel free to let us know.

Having an integration candidate will also help with the effort on defining the right abstractions and documentation.

Thank you!

from chainloop.

Hi @garethr. We just wanted to let you know that we've put in place a new plugins sdk and documentation. This allows anybody to add a fan-out integration in an encapsulated manner. Note: plugins are not fully decoupled from there core yet, we will get there :) #195

We also expect to have a build a plugin example #204 published sometime in the future, but in the meantime, is there any integration endpoint/use-case that you could think it would be interesting to implement? We'd happy to give it a crack.

from chainloop.