Giter VIP home page Giter VIP logo

apicurio-epics's People

Contributors

ericwittmann avatar

Watchers

avatar avatar avatar avatar avatar avatar

apicurio-epics's Issues

Dynamic Storage Maintenance mode

This issue is for tracking and discussion of issues related to maintenance of RHOSR, for example during database upgrades.

Topics to triage:

  • Registry readiness probe causes pods to restart when losing DB connection
  • Implement dynamic maintenance window property

Resources:

Address PII issues in Apicurio cloud apps (e.g. Operate First)

We have a number of cloud services in the Apicurio community that do not currently handle PII warnings in a compliant way. We also do not require the user to agree to any Terms and Conditions, which limits how we can use their data.

Keycloak can support the signing of T&C. Other aspects of privacy/cookie warnings may need to be handled in other ways.

Link to Keycloak docs: https://www.keycloak.org/docs/latest/server_development/#_auth_spi

Migrate all UI apps as ConsoleDot apps (deprecate app-services-ui)

The following repositories need to be rewritten as pure Insights (consoleDot) apps:

  • srs-ui (RHOSR control plane)
  • ads-ui (API Designer)

I think each of these should become pure Insights applications that statically pull in UI components from the upstream repositories. We should move the real UI components into upstream and then export them to be used by the Insights apps. This is the easiest way to share functionality between upstream and downstream, in my opinion.

I don't think using Federated Modules to share e.g. data plane components with srs-ui is a winning strategy. It allows us to separately deploy changes to the data plane UI, but I don't know why we would care about that, since both the control plane and data plane UIs are owned by our team.

References

Dynamic configuration properties in Registry (per-tenant)

This epic is about implementing per-tenant dynamic configuration properties in Apicurio Registry.

Tasks in this epic

Managed API Designer (Phase 1+)

Possible tasks we can tackle prior to Phase 2 availability:

  • API Viewer: display an API/Schema using an API renderer for the specific type. The viewer would take, as input, a unique identifier of an API (URL, API design ID, etc), load the content, render the content using a type-specific renderer. Start with ReDoc/SwaggerUI for OpenAPI.
  • Template management - this could be done in Phase 1
  • Branching/Merging/Versioning - allow more advanced content management by enabling branches and versioning in the UI
  • Custom validation rules
  • SDK code generation

Update support for core artifact types (JSON Schema, OpenAPI, etc) (Data Models 2.0)

This epic is all about improving the support we have for our core artifact types. These core types include:

  1. OpenAPI
  2. AsyncAPI
  3. JSON Schema
  4. Avro
  5. Protobuf

These core types have different versions (e.g. OpenAPI has versions 2.0, 3.0, 3.1). We do not currently support all versions of all types.

Tasks in this epic:

Improved Artifact References support (UI, etc) in Registry

We support references in registry already, but really only in the API. We have some good ideas to improve support for references. Some ideas:

UI support

Add references in the UI, for display and when adding/updating artifacts.

Here is the UX design iterations

CLI support

We have no designs for this, but it sure would be nice. There's probably some overlap between what we can do with the maven plugin and what we can do with the CLI (regarding registration of a collection of self-referencing local files).

De-referencing support

It would nice to have an option in the REST API to dereference the content when downloading it. This could make it much easier for consumers that don't support external references. Challenge: this needs to be implemented for each artifact type. We currently have this implemented in OpenAPI and AsyncAPI only.

Extended maven plugin support

We have basic maven plugin support now. But it would be really nice to be able to just point the maven plugin to a bunch of repository files (that reference each other) and have it figure out how to properly add the content to the registry including appropriate reference mappings.

Tasks in this Epic

[PINNED] Registry Operator upkeep

I need to reserve a chunk of time to implement several requested features for the Operator and publish a new release.

Tasks in this epic may change based on priority & time:

Fleet Shard architecture for RHOSR (aka RHOSR Enterprise)

RHOSAK is currently working on support for allowing customers to provision a Kafka cluster on their own OSD (not yet on customer deployed OpenShift clusters). We can assume this might become a requirement for RHOSR as well.

This task is to explore and document an architecture for how this could work for us. What are the system interactions? Component and/or architecture diagrams, etc.

Migrate from Sentry to GlitchTip

Users of Service Delivery's Sentry deployment,

Please be advised that Service Delivery's deployment of Sentry will be decommissioned in 2023. Our target date for decommissioning Sentry is April 30, 2023. Engineering teams that will continue to need error-tracking functionality are asked to start thinking about - and planning for - migrating from Sentry to Glitchtip. At the end of this email, please find a list of resources that describes how to transition from Sentry to Glitchtip.

If you have questions or concerns about any of this, please post a question in the #sd-app-sre Slack channel, or email me directly.

Glitchtip Transition Resources:

Task: https://github.com/Apicurio/apicurio-tasks/issues/196
Some analysis from another team: https://issues.redhat.com/browse/MGDSTRM-10346

Tasks in this Epic

Apicurio Registry v3 ideas

NOTE: this epic is deprecated

Please use this instead: https://github.com/orgs/Apicurio/projects/6/views/1 (work in progress)

A list of breaking changes to consider when we start planning for v3:

Cleanup/Refactoring

  • Remove the IBM Compatibility API
  • Remove the Confluent Compatibility API v6.
  • Remove Apicurio Registry REST API v1
  • Standardize naming and clean up config properties
  • #57
  • Remove dynamic logging API
  • Apicurio/apicurio-registry#3618

Features/Improvements

Operator

  • Rewrite in Java
  • Apicurio/apicurio-registry#3472
  • Align version of Apicurio Registry, Apicurio Registry Operator and CSV to the single upstream version. This avoids confusion we've seen from the customers when installing Service Registry Operator. The first two can be aligned by our decision to keep the new Operator code in the Registry repository. The CSV must be able to handle situations where the Operator and Operand versions do not change, such as respins, but we can define a versioning approach for these cases, such as an additional number, e.g. 3.0.0-r1, 3.0.0-r2, ...

Notes/Ideas

  • Define behavior around deleting artifacts/versions. Do we want to guarantee that content cannot be deleted, only updated? Consider artifact states as well.
  • We also have an option to change in a non-backwards compatible way:
    • KafkaSQL messages
    • SQL table structure
    • Schema resolver / serde interfaces, there are some deprecated classes there
  • Use common-app-components, maybe even extract some more existing features to there.
  • Cleanup of RegistryStorage interface and related concepts. For example:
    • We have a separate database version entity, but RegistryStorage interface does not have a separate public method for creating versions. They can be only created using the createArtifact* method. This should be separated, and we should have a higher-level layer that does the current functionality of createArtifact*. There is a private method createArtifactVersion that IMHO should not be private. Consider same with content.
    • Split colums such as labels and properties into separate tables, instead of using string serialization (see SqlUtil.serializeProperties method).
    • Separate or consolidate artifact and version metadata, e.g. updateArtifactVersionMetaData takes EditableArtifactMetaDataDto. Consider having a separate metadata table for metadata.
  • Consider having canonical conceptual model classes, available in a separate maven artifact. Review the current DTOs, together with the RegistryStorage interface, but this model can be wider and include classes currently in the common module.
  • Consider using the SPI module decomposition architecture (we are using it somewhat in the common app components). Define a separate SPI module for a given feature that defines interfaces that need to be implemented by a provider of such feature. Then the app/core module can use CDI to find if/which implementors are on the classpath, and possibly provide default implementations. This alleviates strong module dependencies. For example, define an SPI for artifact type providers, and add them to the classpath as needed instead of the app module directly depending on apicurio-registry-schema-util-* via apicurio-registry-schema-util-provider. This can also be done for storage implementations.
  • Review and cleanup of exceptions, related to "Improve error user experience" below.
  • When currently updating (editable) artifact metadata, we also behind the scenes update metadata of the latest artifact version. This happens because artifact metadata includes labels and properties, but they have a globalID as a foreign key, which requires attaching them to the latest version of the artifact. This does not seem to be correct, since artifact should conceptually have it's own set of metadata. (Maybe each version inheriting them when returned to user.)
  • Institute ordering on artifact versions. Currently, we only know which version is the latest. We have a workaround to get an ordering by looking at globalIDs (assuming they are monotonic), but this is error-prone. We should have a predecessor field in every version, to be able to reconstruct the ordering. This will also facilitate git-like versioning (i.e. allow for git-like version trees).

[PINNED] Community engagement/improvement

Blog Posts

Ideas

  • Link to Red Hat services from Apicurio community website. Make this explicit and transparent/clear.
  • Create a realistic demo application that we deploy to production

Support semver for version management in Registry

  • Enable automatic reordering of versions based on their semver ordering. This would enable uploading versions out of the current insertion order
  • Implement compatibility rules support based on whether a major, minor or patch component has changed.
  • Extend the /latest endpoint to match a specific version template (e.g. latest version where the major component is 2, e.g. /latest?match="1.x".

Improved application metrics

Registry:

  • Add measurement for RHSSO call time to prometheus
  • Add measurement for database call time to prometheus
  • Create standard Grafana dashboards

Managed API Designer (Phase 2+)

References

Tasks in this Epic

  • Studio Fleet Manager (Jakub)
  • Integration layer (Jakub)
  • Common Tenant Manager (Enda)
  • Bootstrap Studio back-end (Enda)
  • SQL persistence layer synergy (Carles, Eric)
  • Collaborative editing (Carles)
  • Branching/Merging in Java - Research (Eric)
  • Structured Merging of JSON - Research (Jakub?/Eric?)
  • Security: granular access/permisions (All)
  • References
  • Validation/Compatibility/Semver?

[PINNED] Apicurio blog posts

This epic is "pinned" and tracks the blog posts we would like to write. For each blog post we should create a GH issue in the apicurio-tasks repo and also add a link to it below.

Tasks in this epic

Red Hat Integration Service Registry 2.3 (Q3 2022)

In Q3 of 2022 (roughly mid/end of September, 2022) we intend to release version 2.3 of RHI-Service Registry. This will require the following tasks to be completed:

  • Cut an upstream release of Apicurio Registry 2.3.0.Final
  • Update upstream documentation to reflect the new version
  • Implement additional QE tests for new features:
    • Artifact references
    • New authentication/authorization options
    • Dynamic global configuration options
    • Artifact properties (UI)
    • Import/export (UI)
  • Create productized build of RHI-Service Registry 2.3
  • Update RHI-Service Registry Operator with any required changes
  • Complete QE testing/signoff

Engineering tasks/issues (nice to have)

Deferred post-2.3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.