Feature Request

Currently, the azure-eventgrid module is not dealing with event framework registering an EventSubscriber.

Which Areas Would Be Affected?

extensions

Why Is the Feature Desired?

To spread the event framework use

Solution Proposal

Type of Issue

improvement

Checklist

• assigned appropriate label?
• Do NOT select a milestone or an assignee!

Feature Request

Connector currently has env EDC_BLOBSTORE_ENDPOINT_TEMPLATE with default value https://%s.blob.core.windows.net where placeholder %s get replaced during runtime with azure account name.

Currently Azure storage has a single domain (unlike S3), so it works. But in future, if supports multiple domain, then transfer from one domain to another will not work.

Which Areas Would Be Affected?

Let's say we want to test a Azure to Azure transfer between two participant. Each participant deploys there own Azurite locally.
It will have different domains, such as alice-azurite:10000 and bob-azurite:10000. Hence it doesn't work which forces us to use to share a single azurite deployment for both participants.

Why Is the Feature Desired?

Solution Proposal

Just Like Technology-Aws, Technology-Azure should also support endpoint_override option in transfer_process API.

Bug Report

Describe the Bug

When the transfer request needs to handle several files, it does not use sharedKey to list all the files of a specific prefix, even when it does exist. The method expects the credentials to be stored with a hard-coded pattern key that might not correspond to the value in keyName.

Expected Behavior

All authentication strategies need to be consistent, this implies that listContainerFolder makes uses of the sharedKey if exist, instead of trying to resolve the key again.

The behavior was reported in test addition done in by this tractusx-edc PR

Feature Request

Improve the AzureVault#sanitize method, so that it replaces all disallowed characters with allowed characters. Pls check the official documentation

Which Areas Would Be Affected?

AzureVault

Why Is the Feature Desired?

Users could attempt to store secrets with disallowed characters, so this feature would contribute to improve fault tolerance.

Solution Proposal

• if the secret key does not start with a letter, prefix it with "x-"
• use String.replaceAll() in conjunction with a regex to replace every character that does not match [a-zA-Z0-9]

name: Adoption Request
about: Submit your feature to the project
title: 'Adoption Request'
labels: 'adoption'
assignees: ''

Adoption Request

Thank you for wanting to contribute to the project! We are very happy to see the functionalities of the EDC being extended. Providing this open-source is a great opportunity for others with similar requirements and to avoid additional work.

For any details about the guidelines for submitting features, please take a look at the contribution categories.

General information

Please provide some information about your project or code contribution.

If you choose to be referenced as a "friend", these will be added to the known friends list.
If you choose to add your feature as a core EDC component, links to your current code and correlated issues, discussions, and pull requests are of great importance.

Adoption level

Next, please tell us what level of adoption you intend to provide. (pick only one)

• Reference a feature as "friend"
• Incorporate a feature as core EDC component

Adoption in EDC core

If you chose to add your feature as a core EDC component, please answer the following questions.

Why should this contribution be adopted?

Please argue why this feature must be hosted upstream and be maintained by the EDC core team.
The foundation or core feature is already maintained by the EDC core team, but it is focused on the old Azure Storage account, specifically it does not support directories of the successor service, Azure Data Lake Gen2, and it would be most likely of interest to all parties integrating the EDC on Azure to use this feature, as it has especially a cost impact, due to the possibility to selectively scan the Azure Data Lake.

Could it be achieved with existing functionality? If not, why?

If there is any existing code that can achieve the same thing with little modification, that is usually the preferable way for the EDC core team. We aim to keep the code succinct and want to avoid similar/duplicate code. Make sure you understand the EDC code base well!
Yes, the change is by itself only a little modification to the existing Azure Blob Storage extension.

Are there multiple use cases or applications who will benefit from the contribution?

Basically, we want you to motivate who will use that feature and why, thereby arguing the fact that it is well-suited to be adopted in the core code base. One-off features are better suited to be maintained externally.
As this contribution will help every company integrating the EDC on Azure, it is most likely that multiple use cases and application will benefit from this change.

Can it be achieved without introducing third-party dependencies? If not, which ones?

EDC is a platform rather than an application, therefore we are extremely careful when it comes to introducing third party libraries. The reasons are diverse: security, license issues and over all JAR weight, just to mention a few important ones.
No, it cannot be achieved without third-party dependencies, but these dependencies are already part of the official "Technology_Azure" repository.

Would this feature limit platform independence in any way? If so, how and why?

Features that do not work well in clustered environments are difficult to adopt, since EDC is designed from the ground up to be stateless and clusterable. Similarly, features, that have dependencies onto certain operating systems are difficult to argue.
This question does not apply, as the nature of this change wants to address a specific platform.

Is it going to be a self-contained feature, or would it cut across the entire code base?

Features that have a large impact on the code base are very complex to thoroughly test, they have a high chance to destabilize the code and require careful inspection. Self-contained features on the other hand are easier to isolate and test.
The feature is self-contained as an extension.

Currently the Azure Vault hard codes the DNS endpoint and does not allow to set this in the configuration of the connector. This makes it impossible to black-box test a connector image

.vaultUrl("https://" + keyVaultName + ".vault.azure.net")

.vaultUrl("https://" + keyVaultName + ".vault.azure.net")

The request is to change this to something similar as it is in HashicorpVault:

https://github.com/eclipse-edc/Connector/blob/63a73ec4b585253727a5c56cf9a7e51045747dee/extensions/common/vault/vault-hashicorp/src/main/java/org/eclipse/edc/vault/hashicorp/HashicorpVaultExtension.java#L38

that allows to change the entire endpoint.

Any thoughts on this?

Matthias Binzer

Feature Request

Remove the status checker following the upstream eclipse-edc/Connector#3448

Bug Report

Describe the Bug

During a multi object transfer process, when the DataSource is not AzureStorage, an issue arises if the Consumer defines a blobName in the destination. In this scenario, all objects are overwritten using the same file name, resulting in data loss or unintended overwriting. This occurs due to the reliance of the AzureStorageDataSinkFactory on DataSource specifics, particularly the blobPrefix property, which is further utilized to determine object naming, if the property is present it would mean it is a multi object transfer, but it is true only if the DataSource is AzureStorage.

AzureStorageDataSinkFactory:

AzureStorageDataSink:

The behavior was reported in test addition done in by this tractusx-edc PR.

Feature Request

Caused by this bug: eclipse-edc/Connector#2923
We had to provide a nextForState query that's able to filter by type.
This was not implemented for CosmosDB due to time constraints.

Which Areas Would Be Affected?

CosmosContractNegotiationStore

name: Adoption Request
about: Submit your feature to the project
title: 'Adoption Request'
labels: 'adoption'
assignees: ''

Adoption Request

Thank you for wanting to contribute to the project! We are very happy to see the functionalities of the EDC being extended. Providing this open-source is a great opportunity for others with similar requirements and to avoid additional work.

For any details about the guidelines for submitting features, please take a look at the contribution categories.

General information

Please provide some information about your project or code contribution.

If you choose to be referenced as a "friend", these will be added to the known friends list.
If you choose to add your feature as a core EDC component, links to your current code and correlated issues, discussions, and pull requests are of great importance.

Adoption level

Next, please tell us what level of adoption you intend to provide. (pick only one)

• Reference a feature as "friend"
• Incorporate a feature as core EDC component

Adoption in EDC core

If you chose to add your feature as a core EDC component, please answer the following questions.

Why should this contribution be adopted?

Please argue why this feature must be hosted upstream and be maintained by the EDC core team.
The foundation or core feature is already maintained by the EDC core team. Currently, the extension only supports authentication by providing a shared key. Explicit credential management is companies implies efforts and costs, adhering to security / compliance rules like key rotation. With a few lines of code, a huge security and operational efficiency can be achieved by supporting a standard Microsoft feature named "Federated Identity".

Could it be achieved with existing functionality? If not, why?

If there is any existing code that can achieve the same thing with little modification, that is usually the preferable way for the EDC core team. We aim to keep the code succinct and want to avoid similar/duplicate code. Make sure you understand the EDC code base well!
Yes, the change is by itself only a little modification to the existing Azure Blob Storage extension.

Are there multiple use cases or applications who will benefit from the contribution?

Basically, we want you to motivate who will use that feature and why, thereby arguing the fact that it is well-suited to be adopted in the core code base. One-off features are better suited to be maintained externally.
I strongly assume that this is the case, as the general topic of managing credentials on a platform like Azure is an issue for all companies.

Can it be achieved without introducing third-party dependencies? If not, which ones?

EDC is a platform rather than an application, therefore we are extremely careful when it comes to introducing third party libraries. The reasons are diverse: security, license issues and over all JAR weight, just to mention a few important ones.
No, it cannot be achieved without third-party dependencies, an Azure Identity library has to be added, but that would be just one additional Azure library amongst many which are already present.

Would this feature limit platform independence in any way? If so, how and why?

Features that do not work well in clustered environments are difficult to adopt, since EDC is designed from the ground up to be stateless and clusterable. Similarly, features, that have dependencies onto certain operating systems are difficult to argue.
This question does not apply, as the nature of this change wants to address a specific platform.

Is it going to be a self-contained feature, or would it cut across the entire code base?

Features that have a large impact on the code base are very complex to thoroughly test, they have a high chance to destabilize the code and require careful inspection. Self-contained features on the other hand are easier to isolate and test.
The feature is self-contained as an extension.

name: Adoption Request
about: Submit your feature to the project
title: 'Adoption Request'
labels: 'adoption'
assignees: ''

Adoption Request

Thank you for wanting to contribute to the project! We are very happy to see the functionalities of the EDC being extended. Providing this open-source is a great opportunity for others with similar requirements and to avoid additional work.

For any details about the guidelines for submitting features, please take a look at the contribution categories.

General information

Please provide some information about your project or code contribution.

If you choose to be referenced as a "friend", these will be added to the known friends list.
If you choose to add your feature as a core EDC component, links to your current code and correlated issues, discussions, and pull requests are of great importance.

Adoption level

Next, please tell us what level of adoption you intend to provide. (pick only one)

• Reference a feature as "friend"
• Incorporate a feature as core EDC component

Adoption in EDC core

If you chose to add your feature as a core EDC component, please answer the following questions.

Why should this contribution be adopted?

Please argue why this feature must be hosted upstream and be maintained by the EDC core team.
The foundation or core feature is already maintained by the EDC core team. The issue with it is that, as that's how it is implemented, data received by the extension sort-of "pop-ups" in the storage account and has to be somehow correlated to what it belongs to. Adding metadata is IMHO the natural way how to solve such topics in context of this existing extension, especially adding e. g. adding the BPN of the sending business partner and adding a correlation id, which can be supplied in the data destination of the transfer request.

Could it be achieved with existing functionality? If not, why?

If there is any existing code that can achieve the same thing with little modification, that is usually the preferable way for the EDC core team. We aim to keep the code succinct and want to avoid similar/duplicate code. Make sure you understand the EDC code base well!
Yes, the change is by itself only a little modification to the existing Azure Blob Storage extension.
The only option without any change at all would be to "misuse" the file name of data received to store a UUID as correlation id, etc., but that is not acceptable.

Are there multiple use cases or applications who will benefit from the contribution?

Basically, we want you to motivate who will use that feature and why, thereby arguing the fact that it is well-suited to be adopted in the core code base. One-off features are better suited to be maintained externally.
I strongly assume that this is the case, as the general topic of "correlation" was already addressed by other companies in the CX weekly meeting. The very idea of correlating data received with the actual request is essential and part of every integration scenario within companies.

Can it be achieved without introducing third-party dependencies? If not, which ones?

EDC is a platform rather than an application, therefore we are extremely careful when it comes to introducing third party libraries. The reasons are diverse: security, license issues and over all JAR weight, just to mention a few important ones.
No, it cannot be achieved without third-party dependencies, but these dependencies are already part of the official "Technology_Azure" repository.

Would this feature limit platform independence in any way? If so, how and why?

Features that do not work well in clustered environments are difficult to adopt, since EDC is designed from the ground up to be stateless and clusterable. Similarly, features, that have dependencies onto certain operating systems are difficult to argue.
This question does not apply, as the nature of this change wants to address a specific platform.

Is it going to be a self-contained feature, or would it cut across the entire code base?

Features that have a large impact on the code base are very complex to thoroughly test, they have a high chance to destabilize the code and require careful inspection. Self-contained features on the other hand are easier to isolate and test.
The feature is self-contained as an extension.

Adoption Request

Thank you for wanting to contribute to the project! We are very happy to see the functionalities of the EDC being extended. Providing this open-source is a great opportunity for others with similar requirements and to avoid additional work.

For any details about the guidelines for submitting features, please take a look at the contribution categories.

General information

Please provide some information about your project or code contribution.

If you choose to be referenced as a "friend", these will be added to the known friends list.
If you choose to add your feature as a core EDC component, links to your current code and correlated issues, discussions, and pull requests are of great importance.

Adoption level

Next, please tell us what level of adoption you intend to provide. (pick only one)

• Reference a feature as "friend"
• Incorporate a feature as core EDC component

Adoption in EDC core

If you chose to add your feature as a core EDC component, please answer the following questions.

Why should this contribution be adopted?

Please argue why this feature must be hosted upstream and be maintained by the EDC core team.
The foundation or core feature is already maintained by the EDC core team, but it supports only transfer of single blob. It does not support like for example AWS extension does - registering source data address as 'folder' by specifying blob path prefix. It would be most likely of interest to all parties integrating the EDC on Azure to use such feature.

Could it be achieved with existing functionality? If not, why?

If there is any existing code that can achieve the same thing with little modification, that is usually the preferable way for the EDC core team. We aim to keep the code succinct and want to avoid similar/duplicate code. Make sure you understand the EDC code base well!
Yes, the change is by itself only a little modification/addition to the existing Azure Blob Storage extension.

Are there multiple use cases or applications who will benefit from the contribution?

Basically, we want you to motivate who will use that feature and why, thereby arguing the fact that it is well-suited to be adopted in the core code base. One-off features are better suited to be maintained externally.
As this contribution will help every company integrating the EDC on Azure, it is most likely that multiple use cases and application will benefit from this change.

Can it be achieved without introducing third-party dependencies? If not, which ones?

EDC is a platform rather than an application, therefore we are extremely careful when it comes to introducing third party libraries. The reasons are diverse: security, license issues and over all JAR weight, just to mention a few important ones.
No, it cannot be achieved without third-party dependencies, but these dependencies are already part of the official "Technology_Azure" repository.

Would this feature limit platform independence in any way? If so, how and why?

Features that do not work well in clustered environments are difficult to adopt, since EDC is designed from the ground up to be stateless and clusterable. Similarly, features, that have dependencies onto certain operating systems are difficult to argue.
This question does not apply, as the nature of this change designedly address a specific platform.

Is it going to be a self-contained feature, or would it cut across the entire code base?

Features that have a large impact on the code base are very complex to thoroughly test, they have a high chance to destabilize the code and require careful inspection. Self-contained features on the other hand are easier to isolate and test.
The feature is self-contained as an extension.

Feature Request

Currently privateProperties for assets are only supported for the in-memory and postgreSQL storage.
The implementation for cosmos DB is still missing.

Which Areas Would Be Affected?

asset-index-cosmos

Why Is the Feature Desired?

To enable full support for privateProperties in assets.