asyncapi / modelina Goto Github PK

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

Cannot push to the Git repository.

semantic-release cannot push the version tag to the branch master on the remote Git repository with URL https://x-access-token:[secure]@github.com/asyncapi/generator-model-sdk.git.

This can be caused by:

• a misconfiguration of the repositoryUrl option
• the repository being unavailable
• or missing push permission for the user configured via the Git credentials on your CI environment

Good luck with your project ✨

Your semantic-release bot 📦🚀

At the moment we don't have generator (like TypeScript generator - https://github.com/asyncapi/generator-model-sdk/blob/master/src/generators/typescript/TypeScriptGenerator.ts) instance inside the renderers. This will facilitate many things related to rendering models.

Use cases: for example render enum type inside class type in Java. At the moment we must do workaround like here https://github.com/asyncapi/java-spring-template/pull/121/files#diff-812b0ef67b54ce5586eb2192840e7dbdb0c16608074f37f67a896fe733ff9436R49 to achieve inline enum in class.

Describe the bug

There are certain keywords that are reserved. This is different between languages, we need to make sure we dont render such properties as-is.

How to Reproduce

For example, in Java, you cannot have a property named enum.

Expected behavior

I expected the library to handle any property names.

Describe the bug

Since CommonModel items can be an array the function should handle that and not assume items are only one type.

How to Reproduce

Location of function: https://github.com/asyncapi/generator-model-sdk/blob/9dec1e58ec5da21e8dbd80fc44075c53f156b830/src/models/CommonModel.ts#L232
Remember to change the tests as well.

Caught by @magicmatatjahu here: #183 (comment)

Expected behavior

Should handle CommonModel | CommonModel[] and return accurately.

Implement C# generator and renderers like in TS or Java.

Generator should have possibility to define presets for class, enum type.

Additional: create presets for serialisation/converters from json.

Reason/Context

We need to ensure that this library is 100% JSON Schema draft 7 compliant since this is the main input source of the library.

Description

We need to use their test compatibility suite. I suggest we wrap any of their schemas in an object property to force the library to generate models.

Something similar to:

{
    type: "object",
    properties: {
        "testProp": <suite test schema>
    }
}

I am not sure how we can ensure the correct output though🤔 Any ideas?

generator-model-sdk can be confusing for someone in AsyncAPI community and also for people outside the community. It doesn't need the generator to work, it's a standalone library like QuickType, so we should change name of the repo and project. Any ideas?

My proposition is Modelina. It is a polish word for modelling clay. There is possible to form something (models) out of modeling clay (input data) - like in programming 🤣 . And the model is also in the name :)

Describe the bug

When I fixing rendering the tuple for TS, I found the bug.

At the moment given schema:

{ "type": "array", "items": [{ "type": "string" }, { "type": "number" }] }

produces given CommonModel:

CommonModel {
  originalSchema: Schema { type: 'array', items: [ [Schema], [Schema] ] },
  type: [ 'string', 'number' ]
}

and this is not true, because it's a tuple, not an array of unions. We should fix it.

Reason/Context

Currently we do not iterate schema dependencies. We should iterate them and combine properties/items/types etc together.

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "credit_card": { "type": "number" }
  },
  "required": ["name"],
  "dependencies": {
    "credit_card": {
      "properties": {
        "billing_address": { "type": "string" }
      },
      "required": ["billing_address"]
    }
  }
}

should merge it together to give the model:

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "credit_card": { "type": "number" },
    "billing_address": { "type": "string" }
  },
  "required": ["name"]
}

Reason why billing_address is not added to the list of required properties, is that it is a runtime requirement that we cannot do anything about when rendering the model.

see more here: http://json-schema.org/understanding-json-schema/reference/object.html#schema-dependencies

Reason/Context

It should be possible to render pattern properties.

Example: #43 (comment)

Blocked by #153

Reason/Context

Because this library should not be directly associated with the generator but a separate library we should introduce a CLI for the library.

Description

Be aware that this CLI might be part of a larger CLI and should be implemented with extensibility in mind.

Related issues

#132

At the moment we have only one way flow in presets. It means that render starts from defaultPreset, goes through every custom preset and return to output final string. Sometimes we need to adjust to final part of model something after rendering the rest of custom presets:

e.g. we have CUSTOM_PRESET_1, CUSTOM_PRESET_2, in CUSTOM_PRESET_1 I wanna run first CUSTOM_PRESET_2 and then add something to final string by CUSTOM_PRESET_1. Of course I can change order of CUSTOM_PRESET_2 with CUSTOM_PRESET_1 but some actions cannot be achieved by changing order - we wanna render some property, go through by every preset for property, but in final string, render something additional above the property... in this case we have problems:

• as a preset creator I don't know how many presets there are when rendering.
• sometimes subsequent presets may render the final model incorrectly - like here https://github.com/asyncapi/java-spring-template/pull/121/files#diff-812b0ef67b54ce5586eb2192840e7dbdb0c16608074f37f67a896fe733ff9436R110 - I must add @Valid annotation to property which uses inline enum - by directional preset I can avoid repetitive boilerplate.
• related to the second point. Some presets can be completely overwritten and to run flow from beginning you need to run the preset (like property) in the another preset, but this can result in circular recursion.

Of course bidirectional preset flow doesn't fix all problem related to overwriting presets result - some preset must be render as last - but we can avoid some problems.

Maybe bidirectional preset flow isn't a good thing, how do you think about that? Maybe another suggestion to resolve above problems?

We should document how user/dev should use our model-sdk in templates - here we should have examples with React and Nunjucks templating - in library, or in normal app.

Documents should contain also complex cases like:

• how to use pass options to generators.
• how to use presets in given language - with overriding case, concatenation etc.

Also we should update main Readme.md to provide information what user/dev should do to use out package and small introduction to package like in react-sdk package.

Describe the bug

mergeCommonModels doesn`t merge required field

How to Reproduce

use allOf with objects with required properties

Expected behavior

mergeCommonModels does merge required fields

Reason/Context

It would be awesome to take TS code and be able to output models in any other language.

Description

There already exist libraries we can leverage to minimize the effort to integrate this where we could use the already existing JSON Schema process or create a custom one.

Link shared by Kevin Swiber: https://github.com/YousefED/typescript-json-schema

Reason/Context

Currently, whenever types are simplified and come across a schema of type false it throws an error and the library won't generate anything which is far from ideal.

https://github.com/asyncapi/generator-model-sdk/blob/cc42b3ae7753044c173cbdd16cfa4f9fb7cb2c34/src/simplification/SimplifyTypes.ts#L16

Description

I suggest we just ignore that specific model entirely.

Context

I have a function saveProject which save a asyncApiMessage as a document to a database.
This document is message wrapped in Envelope with additional properties.
A message looks like this:

export interface MessageProjectCreated { //this is what is generated from generator-model-sdk everything else is my code
   eventName: "tms.event.project.created";
};
export const message: MessageProjectCreated = {
   eventName: "tms.event.project.created",
};

My document look like this:

export enum EventName {
  ProjectCreated = "tms.event.project.created",
  ...
};
export interface Envelope<T> {
   name: EventName;
   payload: T;
};

export const document: Envelope<MessageProjectCreated> = {
   name: message.eventName,
   payload: event,
};

Reason#1

My problem with this is that typescript treat generated "tms.event.project.created" as a string not as part of EventName.
To make this work I had to replace Envelope with this.

export interface Envelope<T> {
   name: EventName | string; // a hacky way to make use of generated types
   payload: T;
};

Reason#2

I have to declare EventName enum by my own which can lead to unconsistency between event names from specification and enum written by me.

Description

I need a way to create enum in asyncapi and a way to specify field type as enum element.
So something like this: (I modified json below from this answer in SO)

{
  "$id":"MessageProjectCreated",
  "properties": {
    "eventName": {
      "$ref": "http://example.com/some_schema.json#/definitions/EventName/tms.event.project.created"
    }
  }
}

{
  "$id": "http://example.com/some_schema.json",
  "definitions": {
    "EventName": {
      "enum": [ "tms.event.project.created"]
    }
  }
}

should generate something like this.

export enum EventName {
  ProjectCreated = "tms.event.project.created",
};
export interface MessageProjectCreated {
   eventName: EventName.ProjectCreated;
};

Describe the bug

We are expecting the input JSON Schema draft 7 to be a document that can validate correctly, but we do nothing when this is not the case.

Example: If a schema uses combination schemas that each contain the same property name, it will try to merge the two common models. However, if it is from an allOf combination schema, it should "fail" somehow if conflicts occur. Since the input JSON Schema could never validate any data we should consider it an invalid input.

How to Reproduce

Given the JSON Schema:

{
  "allOf": [
    {
      "type": "object",
      "properties": {
        "somePropName": {
          "type": "number"
        }
      }
    },
    {
      "type": "object",
      "properties": {
        "somePropName": {
          "enum": [
            "residential",
            "business"
          ]
        }
      },
      "required": [
        "type"
      ]
    }
  ]
}

It would create the simplified model:

{
  "type": "object",
  "properties": {
    "somePropName": {
      "type": [
        "number",
        "string"
      ]
    }
  }
}

Which is incorrect. However, it is the input that is incorrect, since the JSON Schema can never validate the input.

Expected behavior

I would expect the model generator to fail the generation, or handle invalid inputs somehow.

Reason/Context

We need to find a replacement name for this function since it is not clear what it does.

https://github.com/asyncapi/generator-model-sdk/blob/0a8f5e3093620a4ff064361c896fc6955ad136e0/src/models/CommonModel.ts#L186

Description

What it means is any direct/immediate/nearest dependencies of models. An example could modelA have the property prop1 that refers to another model - modelB. The immediate dependencies for modelA is then modelB because it is referenced through prop1. If modelB had any dependencies on its own, modelA would not return those since it is not the immediate dependencies.

@magicmatatjahu suggested getNearestDependencies as a replacement which I agree could be a good solution, any other suggestions?

We don't support simplification for metadata fields like minItems, minimum, maxLength etc. in combined schemas like anyOf, oneOf, allOf. This issue is a tracker for each missed field.

• Number/Integer:
  • minimum
  • exclusiveMinimum
  • maximum
  • exclusiveMaximum
  • multipleOf
• String:
  • minLength
  • maxLength
  • pattern
• Array:
  • minItems
  • maxItems
  • uniqueItems
• Object:
  • minProperties
  • maxProperties
• All
  • examples

Please update if something was not included.

As in title. At the moment, if someone change something in simplifiers and processors must update (sometimes) 20-30 test cases. It wouldn't be a problem, but the tests are written as integration tests that each test that tests the results of a function with content stored in separate files. Some files are big enough that it is tiresome. Each test should be an individual test that tests only one part of the scheme, not the whole.

Reason/Context

Currently we ignore warnings but it needs to be addressed. Especially the complexity check. After that we need to force no warnings.

Describe the issue

As the comment states here:
https://github.com/asyncapi/generator-model-sdk/blob/cc42b3ae7753044c173cbdd16cfa4f9fb7cb2c34/src/simplification/Utils.ts#L7

We need to refactor a bit to better determine if a CommonModel is a simple model, generally whenever the model type != object, or an independent model which should be generated as such.

The reason why we cannot use the new check yet is that this function is used before the properties extend and properties are even sat on the model.

Also reflect on whether we can do modify the check to work with the current implementation without having to refactor.

By default, we simplify JSON schema into multiple classes such that:

{
  "allOf": [
    {
      "type": "object",
      "properties": {
        "street_address": {
          "type": "string"
        },
        "country": {
          "type": "string"
        }
      }
    },
    {
      "type": "object",
      "properties": {
        "street_address2": {
          "type": "string"
        },
        "country2": {
          "type": "string"
        }
      }
    }
  ]
}

should render 3 classes, root which extends rootAllOf1 and rootAllOf2, and then the later two classes are then separated. But since the renderer is not ready for this we need to switch to defaulting to merging the allOf schemas instead. So no schemas are left behind.

Reason/Context

At the moment we do not directly test the correctness of the output. So when we receive PRs changing the rendering in any way we have no way of knowing whether the output can actually compile.

We already encountered this issue and having something in place to handle this logic would be awesome.

Description

I would suggest we implement a GitHub action per language we output to. For now, this means we would create a workflow for Java, Typescript, and JS.

Questions that need to be answered:

• Should this workflow run for each PR?
• Should we create one global input file which covers all possible values the output can be?
• Say we scale to 10 languages, how much time would that affect the "wait time" for checks to complete?
• What about different versions of languages, should that be tested?

Describe the bug

Library entirely ignore required field in components/schemas while merging required fields.
Probably schemas are not instanced to CommonModel 😢 😞

How to Reproduce

Run parser with those components and check if project interface has id?: string.

components:
  schemas:
    EventEnvelope:
      $id: EventEnvelope
      description: "Envelope shared between messages"
      type: object
      required:
        - eventName
      properties:
        eventName:
          type: string
    
    Project:
      $id: Project
      type: object
      required: [id]
      properties:
        id:
          type: string

  messages:
       someEventName:
        $id: someEventName
        allOf:
          - $ref: "#/components/schemas/EventEnvelope"
          - type: object
            properties:
              data: object
              properties: 
                project: 
                  $ref: "#/components/schemas/Project"

• Screenshots
• Link to GitHub repository with project that has issues
• Files that can help reproduce the issue, like your AsyncAPI file, either paste inside the issue in a code block or in a sharable gist

Expected behavior

project interface has id: string

Workaround

Set required fields everytime you do $ref: "#/components/schemas/Project". It has to be done everywhere you do reference, if not field will not be required.

After first release, we will have only 3 languages which we will support - JS, TS and Java with custom presets, e.g. with Jackson support (Java). We should document how community can contribute to our model-sdk with implementation of new languages and presets. Documents should contain:

• how to extend AbstractGenerator.
• how to extend AbstractRenderer.
• how to write custom renderers and how to connect them to the generator.
• how to write default preset
• how to write custom presets for given language

Also mentioned in #70

• how generator works

Reason/Context

One of the strong features of the CommonModel is the possibility to use extend which contains a list of $ids of the CommonModels that the current model should extend. Therefore we need to figure out how we render this in the different languages

Reason/Context

Considering that we are potentially gonna have parsers written in other languages (not just JS. E.g. https://github.com/asyncapi/parser-go), It makes sense to support an input type that can be used to load schemas generated by any of those.

This will allow using any convenient parser, no matters the language they are implemented on.

This feature request was motivated by this PR review comment.

Description

Ideally, the parsers should be able to generate JSON Schema files so they can be used later by this generator.

We should improve typings in whole project to improve DX:

• improve generic types in AbstractGenerator, AbstractRenderer - #60
• improve generic types in presets: CommonPresets... and in Preset abstract interface to include typings for options for given preset.

While integrating this library into ts-nats-template I got frustrated because my expected outcome was not aligned with the default preset. And this comes down to my basic understanding of it was wrong, the setup is great because it offers many different ways of rendering a model. However, I realized that we need a clear direction in which way we want to take these presets.

As I see it we have 3(+) choices in which our default preset could move towards:

1. The complete model
2. The data transfer model
3. The minimalistic
4. Do you have another suggestion?

The complete model

This model gives you everything in regards to rendering a complete model. Where you have access to not only the properties but additionalProperties, patternProperties, inheritance, etc.

The data transfer model

Optimized for serializing and deserializing data.

The minimalistic

Just the bare minimum properties, basically what we have now.

Questions

• What should the default preset be?
• What presets should we include with this library by default?
  • How many different presets should the library provide? I.e. all 3 before-mentioned presets?

Related issues and PRs

All the following issues/PRs are blocked until we figure out the direction.

• #68 and PR #71
• #107
• #43

The following issues/PRs are somewhat blocked since this could trigger a rewrite unless we are careful.

• #49
• #48
• #47

Reason/Context

Pattern properties should be merged with additional properties
Pattern properties should be included the same way that additionalProperties are. Meaning it should be moved from Schema to CommonSchema.

Implement Python generator and renderers like in TS or Java.

Generator should have possibility to define presets for class, enum type.

Additional: create presets for serialisation/converters from json.

Reason/Context

We should provide a logging interface for users of the library to get notified on internal logs.

Use-case:

• In case two simplified models need to be merged some properties such as $id are overwritten, in these special cases I would like to inform the user that a clash between properties occurred and the output might not contain the expected output.
• As a user I would like to see which models are generated

Description

I suggest we provide a logging interface so the users of the library can decide which logs they would like to do something with.

This could for example include 4 levels:

• Debug
• Info
• Warning
• Error

Describe the bug

Currently, in templates, all you have access to is the uid() function from the AsyncAPI parser for the schema which returns a different name than what the model is generated with. This should not be the case.

Why? Because I cannot change what I have access to, we should not interpret another id if one is already given by another tool.

Reason/Context

If you have your input in a file you have to load your file before using the generator instead of just providing a file path and I feel like the library should be able to handle such inputs.

The files we should accept are YAML and JSON files.

Description

This will be a minor change and improve the overall interaction with the library.

Describe the bug

Our current flow for generating docs is first transpile TypeScript to Javascript, then run jsdoc2md to generate the final Markdown file. See https://github.com/asyncapi/generator-model-sdk/blob/master/package.json#L62.

Documenting interfaces in jsdoc is totally possible thanks to the @interface tag. However, the tag is not supported by TS compiler. See https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html.

That means we cannot generate documentation for interfaces later on with jsdoc2md as those tags are not added ever.

Expected behavior

We somehow generate documentation for interfaces.

I would suggest we re-evaluate the current flow so we can either:

1. Generate the docs directly from TS:
   • https://github.com/microsoft/tsdoc (Standard, but I think it's WIP)
   • http://typedoc.org/
2. Rather we transpile using an alternative like https://github.com/SoftwareBrothers/better-docs#how-it-works

Reason/Context

I created generator which generate types and Joi validators.
I generate validator with code like below.

validators.enjoiOne("MessageJobCreated")

As you can see I must pass string what is a little bit dirty solution.
This string is $id of message from asyncapi yaml file but there is no way to access it from generated types.

Description

I would like to have access to enum with all messages $id`s as key and value.
I guess it may look similar to this one.

export enum AsyncApiMessages {
   MessageJobCreated = "MessageJobCreated" 
}

We should have naming convention for models, it means naming style for model type (included refs) or property. I suggest to have in default option of generator namingConvention object with type and property fields. At the moment developer must change manually name of type or property, e.g. type is called some_class and in Java should be pascalCased - SomeClass - at the moment we must do it inside code, by helpers - finally user of data-model generation cannot change naming style, because it is hardcoded.

My suggestion for signature:

options = {
   // rest of default options
   namingConvention: {
     type: (name: string, model: CommonModel) => string,
     property: (name: string, model: CommonModel) => string,
   }
}

Using in code (preset):

const preset = {
  class: {
    property({ renderer, propertyName, property }) {
      const name = renderer.nameProperty(propertyName, property);
      // logic
    }
  }
}

Implement GoLang generator and renderers like in TS or Java.

Generator should have possibility to define presets for struct, enum and normal type type.

Additional: create presets for serialisation/converters from json.

Describe the bug

non-required properties renderers a wrong signature for encapsulation.

How to Reproduce

Given the following JSON schema

{
	"type": "object",
	"$id": "GeneralReply",
	"properties": {
		"status_code": {
			"type": "integer"
		},
		"status_message": {
			"type": "string"
		}
	}
}

It is rendered to

export class GeneralReply {
  private _statusCode?: number;
  private _statusMessage?: string;

  constructor(input: {
    statusCode?: number,
    statusMessage?: string,
  }) {
    this._statusCode = input.statusCode;
    this._statusMessage = input.statusMessage;
  }

  get statusCode(): number { return this._statusCode; }
  set statusCode(statusCode: number) { this._statusCode = statusCode; }

  get statusMessage(): string { return this._statusMessage; }
  set statusMessage(statusMessage: string) { this._statusMessage = statusMessage; }
}

It is however unable to compile based on the following error messages:

> tsc

src/messages/GeneralReply.ts:13:30 - error TS2322: Type 'number | undefined' is not assignable to type 'number'.
  Type 'undefined' is not assignable to type 'number'.

13   get statusCode(): number { return this._statusCode; }
                                ~~~~~~~~~~~~~~~~~~~~~~~~

src/messages/GeneralReply.ts:16:33 - error TS2322: Type 'string | undefined' is not assignable to type 'string'.
  Type 'undefined' is not assignable to type 'string'.

16   get statusMessage(): string { return this._statusMessage; }
                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~


Found 2 errors.

Describe the bug

see #167 (comment)

Reason/Context

We should simplify additionalProperties

Given schema with dependencies data:

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "credit_card": { "type": "number" }
  },
  "required": ["name"],
  "dependencies": {
    "credit_card": {
      "properties": {
        "billing_address": { "type": "string" }
      },
      "required": ["billing_address"]
    }
  }
}

should be transformed and simplified:

{
  oneOf: [
    {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "credit_card": { "type": "number" },
        "billing_address": { "type": "string" }
      },
      "required": ["name", "credit_card", "billing_address"]
    },
    {
      "type": "object",
      "properties": {
        "name": { "type": "string" }
      },
      "required": ["name"]
    }
  ]
}

Related to #79

Reason/Context

I have had a few use-cases now where I need to generate example JSON based on JSON Schema, and if we can generate the underlying model we should be able to generate an example JSON for that model as well.

Description

It does not have to be fancy just a quick and dirty example will suffice.

Reason/Context

The reason why we should make this change is that the input does not necessarily have be JSON Schema but could be something completely unrelated.

Reason/Context

Fresh into the generator is not easy without some docs explaining what and how it does things to get started.

Describe the bug

We are using flatMap function which is not added before node version 11.

https://github.com/asyncapi/generator-model-sdk/blob/f52a7701d7ca4b89aae1a16940b647bfb69b6bd6/src/helpers/FormatHelpers.ts#L59

What should change

We need to decide if we want to add the requirement to the readme file and package file or whether we should remove this function use. If we decide the later, to what point should we stop being backward compatible?

If we have property as enum like:

state:          { type: "string", enum: ["Texas", "Alabama", "California", "other"] }

then we have rendered in TS:

state: string

but should be:

state: "Texas" | "Alabama" | "California" | "other"

Reason/Context

Currently the CommonModel does include additionalProperties, however they are not rendered as properties in the generator

Questions:

• How should we handle additionalProperties if we already have a property called additionalProperties?