Giter VIP home page Giter VIP logo

asyncapi / modelina Goto Github PK

View Code? Open in Web Editor NEW
271.0 6.0 153.0 18.58 MB

A library for generating typed models based on inputs such as AsyncAPI, OpenAPI, and JSON Schema documents with high customization

Home Page: https://modelina.org

License: Apache License 2.0

TypeScript 98.14% JavaScript 0.89% Dockerfile 0.08% CSS 0.07% Java 0.06% Rust 0.05% PHP 0.03% Kotlin 0.15% C++ 0.04% C# 0.13% Go 0.24% Scala 0.06% Python 0.05%
json-schema asyncapi codegen codegenerator typescript openapi2 openapi3 swagger cpp csharp

modelina's Introduction

AsyncAPI Modelina blackbox pipeline status Coverage Status Project Status: Active – The project has reached a stable, usable state and is being actively developed. Maintenance score Npm latest version License last commit Discussions Website Playground All Contributors

Your one-stop tool for generating accurate and well-tested models for representing the message payloads. Use it as a tool in your development workflow, or a library in a larger integrations, entirely in your control.


Installing Modelina

Run this command to install Modelina in your project:

npm install @asyncapi/modelina

AsyncAPI CLI

If you have the AsyncAPI CLI installed (ONLY support AsyncAPI inputs), you can run the following command to use Modelina:

asyncapi generate models <language> ./asyncapi.json

What Does Modelina Do?

Modelina put YOU in control of your data models, here is how...

Modelina lets you generate data models from many types of inputs
const asyncapi = ...
const jsonschema = ...
const openapi = ... 
const metamodel = ... 
...
const models = await generator.generate(
  asyncapi | jsonschema | openapi | metamodel
);
Use the same inputs across a range of different generators
const generator = new TypeScriptGenerator();
const generator = new CsharpGenerator();
const generator = new JavaGenerator();
const generator = new RustGenerator();
...
const models = await generator.generate(input);
Easily let you interact with the generated models.
  • Want to show the generated models on a website? Sure!
  • Want to generate the models into files? Sure!
  • Want to combine all the models into one single file? Sure!

Whatever interaction you need, you can create.

const models = await generator.generate(input);
for (const model in models) { 
  const generatedCode = model.result;
  const dependencies = model.dependencies;
  const modeltype = model.type;
  const modelName = model.modelName;
  ...
}
Easily modify how models are constrained into the output
const generator = new TypeScriptGenerator({
  constraints: {
    modelName: ({modelName}) => {
      // Implement your own constraining logic
      return modelName;
    }
  }
});
Seamlessly layer additional or replacement code on top of each other to customize the models to your use-case
const generator = new TypeScriptGenerator({
  presets: [
    {
      class: {
        additionalContent({ content }) {
          return `${content}
public myCustomFunction(): string {
  return 'A custom function for each class';
}`;
        },
      }
    }
  ]
});
const models = await generator.generate(input);
Seamlessly lets you combine multiple layers of additional or replacement code
const myCustomFunction1 = {
  class: {
    additionalContent({ content }) {
      return `${content}
public myCustomFunction(): string {
return 'A custom function for each class';
}`;
    },
  }
};
const myCustomFunction2 = {...};
const generator = new TypeScriptGenerator({
  presets: [
    myCustomFunction1,
    myCustomFunction2
  ]
});
const models = await generator.generate(input);

Features

The following table provides a short summary of available features for supported output languages. To see the complete feature list for each language, please click the individual links for each language.

Supported inputs
AsyncAPI We support the following AsyncAPI versions: 2.0.0 -> 2.6.0, which generates models for all the defined message payloads. It supports the following schemaFormats AsyncAPI Schema object, JSON Schema draft 7, AVRO 1.9, RAML 1.0 data type, and OpenAPI 3.0 Schema.
JSON Schema We support the following JSON Schema versions: Draft-4, Draft-6 and Draft-7
OpenAPI We support the following OpenAPI versions: Swagger 2.0, OpenAPI 3.0 and 3.1, which generates models for all the defined request and response payloads.
TypeScript We currently support TypeScript types as file input for model generation
Meta model This is the internal representation of a model for Modelina, it is what inputs gets converted to, and what generators are provided to generate code. Instead of relying on an input processor, you can create your own models from scratch and still take advantage on the generators and the features.

Supported outputs
Java Class and enum generation: generation of equals, hashCode, toString, Jackson annotation, custom indentation type and size, etc
TypeScript Class, interface and enum generation: generation of example code, un/marshal functions, custom indentation type and size, etc
C# Class and enum generation: generation of example code, serializer and deserializer functions, custom indentation type and size, etc
Go Struct and enum generation: custom indentation type and size, etc
JavaScript Class generation: custom indentation type and size, etc
Dart Class and enum generation: json_annotation
Rust Struct/tuple and enum generation: generation of `implement Default`, generate serde macros, custom indentation type and size, etc
Python Class and enum generation: custom indentation type and size, etc
Kotlin Class and enum generation: use of data classes where appropriate, custom indentation type and size, etc
C++ Class and enum generation: custom indentation type and size, etc
PHP Class and enum generation: custom indentation type and size, descriptions, etc
Scala Class and enum generation: custom indentation type and size, descriptions, etc

Requirements

The following are a requirement in order to use Modelina.

Documentation

A feature in Modelina cannot exists without an example and documentation for it. You can find all the documentation here.

Examples

Do you need to know how to use the library in certain scenarios?

We have gathered all the examples in a separate folder and they can be found under the examples folder.

Versioning and maintenance

As of version 1, Modelina has a very strict set of changes we are allowed to do before it requires a major version change. In short, any changes that change the generated outcome are not allowed as it's a breaking change for the consumer of the generated models.

Here is a list of changes we are allowed to do that would not require a breaking change:

  • Adding new features (that do not change existing output), such as generators, presets, input processors, etc.
  • Change existing features, by providing options that default to current behavior. This could be a preset that adapts the output based on options, as long as the API of Modelina and the API of the generated models does not have any breaking changes.
  • Bug fixes where the generated code is otherwise unusable (syntax errors, etc).

Breaking changes are allowed and expected at a frequent rate, of course where it makes sense we will try to bundle multiple changes together.

We of course will do our best to uphold this, but mistakes can happen, and if you notice any breaking changes please let us know!

Because of the number of the limited number of champions, only the most recent major version will be maintained.

Major versions are currently happening at a 3-month cadence (in a similar fashion as the AsyncAPI specification), this will happen in January, April, June, and September.

Development

We try to make it as easy for you as possible to set up your development environment to contribute to Modelina. You can find the development documentation here.

Contributing

Without contributions, Modelina would not exist, it's a community project we build together to create the best possible building blocks, and we do this through champions.

We have made quite a comprehensive contribution guide to give you a lending hand in how different features and changes are introduced.

If no documentation helps you, here is how you can reach out to get help:

Contributors

Thanks go out to these wonderful people (emoji key):

Maciej Urbańczyk
Maciej Urbańczyk

🐛 💻 📖 🤔 🚧 💬 ⚠️ 👀
czlowiek488
czlowiek488

🐛 👀 🤔
Sergio Moya
Sergio Moya

🐛 💻 ⚠️ 📖 👀
Jonas Lagoni
Jonas Lagoni

🐛 💻 📖 🤔 🚧 💬 ⚠️ 👀
Lukasz Gornicki
Lukasz Gornicki

👀 🐛 💻
Arjun Garg
Arjun Garg

💻
Fran Méndez
Fran Méndez

👀
Kanwal Singh
Kanwal Singh

💻
Alejandra Quetzalli
Alejandra Quetzalli

👀 📖
MD SAIF  HUSAIN
MD SAIF HUSAIN

💡 ⚠️ 📖 💻
Sudipto Ghosh
Sudipto Ghosh

💡 ⚠️ 📖
panwauu
panwauu

💻 ⚠️ 💡 📖 🐛
Stefan E. Mayer
Stefan E. Mayer

🐛
Talmiz Ahmed
Talmiz Ahmed

📖 ⚠️ 💡
Marco
Marco

🐛
quadrrem
quadrrem

💻 ⚠️
Kamil Janeček
Kamil Janeček

⚠️ 🐛 💻
mahakporwal02
mahakporwal02

💡 ⚠️ 📖 💻
Debajyoti Halder
Debajyoti Halder

💻 ⚠️ 📖 💡 🚧
Ritik Rawal
Ritik Rawal

📖 💻 ⚠️ 💡
Ishan
Ishan

💻 ⚠️
Samriddhi
Samriddhi

💻 ⚠️ 📖 💡 🚧
Gábor Magyar
Gábor Magyar

💻 ⚠️ 📖 💡
ibernabeudev
ibernabeudev

💻 ⚠️ 💡 📖
Arkadiusz Słowikowski
Arkadiusz Słowikowski

💻 ⚠️ 💡 📖
Willem Gillis
Willem Gillis

💻 ⚠️ 🐛
rmasarovic
rmasarovic

💻 📖 ⚠️ 💡
Owais Hasnath Ahmed
Owais Hasnath Ahmed

💻 ⚠️ 📖
PanMan
PanMan

📖
artur-ciocanu
artur-ciocanu

💻 🐛 💡 📖 ⚠️ 🚧
Cyprian Gracz
Cyprian Gracz

💻 ⚠️ 🐛
Leigh Johnson
Leigh Johnson

💻 ⚠️ 💡 📖 🚧 👀
Nitin Tejuja
Nitin Tejuja

⚠️ 💡
Kenneth Aasan
Kenneth Aasan

💻 ⚠️ 🚧 📖
Amit Kumar Sharma
Amit Kumar Sharma

⚠️ 📖 💡
Andrey Zaytsev
Andrey Zaytsev

💻 💡 📖 ⚠️
Tenshi Codes
Tenshi Codes

🚇
Yushi OMOTE
Yushi OMOTE

🐛 💻
Zbigniew Malcherczyk
Zbigniew Malcherczyk

🐛 🚇 💻 💡 ⚠️ 👀
200Puls
200Puls

💻 ⚠️
Anay Sarkar
Anay Sarkar

💡 💻 ⚠️
Louis Xhaferi
Louis Xhaferi

💻
Sambhav Gupta
Sambhav Gupta

📖 👀 🎨 💻 🐛
Abhay Garg
Abhay Garg

💻 💡 ⚠️ 📖
henrikjon
henrikjon

💻 ⚠️ 📖 💡
Mohammad Yasir
Mohammad Yasir

💻
Savio Dias
Savio Dias

💻 🚇 🐛
Ishaan Shah
Ishaan Shah

💻
Shreyas0410
Shreyas0410

💻
beku-epitome
beku-epitome

💻 👀 ⚠️
Joshua Michael Daly
Joshua Michael Daly

🐛
Daniel KJ
Daniel KJ

💻 ⚠️ 💡 👀
Bhavik Agarwal
Bhavik Agarwal

🎨
Rishi
Rishi

💻 🎨
Rohith Boppey
Rohith Boppey

💻 🎨
Ashish Padhy
Ashish Padhy

💻 ⚠️ 🚇
Jean-François Côté
Jean-François Côté

💻 ⚠️ 💡 📖
Sumant.xD
Sumant.xD

⚠️ 🚇 💻
Aryan Singh
Aryan Singh

💻
Markus Poerschke
Markus Poerschke

💻 ⚠️ 💡 📖
James Moey
James Moey

💻 ⚠️
tomwolanski
tomwolanski

🐛
Kristupas
Kristupas

💻
Devansh-Bhatt
Devansh-Bhatt

⚠️ 🚇
Ansh Pancholi
Ansh Pancholi

💻
Maeght Loan
Maeght Loan

💻 📖
Prince Rajpoot
Prince Rajpoot

💻
harshit mishra
harshit mishra

💻
PeteAudinate
PeteAudinate

💻
jano-petras
jano-petras

💻 📖 ⚠️
Nilkanth Parmar
Nilkanth Parmar

💻 ⚠️ 💡 📖
Ashmit JaiSarita Gupta
Ashmit JaiSarita Gupta

💻 🎨 📖 🚧 ⚠️ 👀
Harshil Jani
Harshil Jani

💻 ⚠️
Mintu Gogoi
Mintu Gogoi

⚠️
Devansh Mahant
Devansh Mahant

📖
Asish Kumar
Asish Kumar

📖
Ankur Singh
Ankur Singh

📖
RowlandBanks
RowlandBanks

💻 ⚠️ 🐛
Moritz Kalwa
Moritz Kalwa

💻 ⚠️ 🐛 📖
Akshit Gupta
Akshit Gupta

💻 🐛 💡 ⚠️ 📖
Athul Tulasidasan
Athul Tulasidasan

💻 💡 📖 ⚠️
souvik
souvik

💻 ⚠️ 💡 📖
Louis-PhilippeGentile
Louis-PhilippeGentile

💻 ⚠️ 📖 🐛
Akhil Jamwal
Akhil Jamwal

💻 ⚠️
Anders Hellerup Madsen
Anders Hellerup Madsen

💻 ⚠️ 🐛
rquinio1A
rquinio1A

💻 💡
gitankit7
gitankit7

💻
Janne Husberg
Janne Husberg

💻 ⚠️ 🐛

This project follows the all-contributors specification. Contributions of any kind are welcome!

modelina's People

Contributors

akkshitgupta avatar allcontributors[bot] avatar artur-ciocanu avatar asyncapi-bot avatar dependabot[bot] avatar derberg avatar devilkiller-ag avatar ferror avatar hashtalmiz avatar jfcote avatar jonaslagoni avatar kaushik-rishi avatar magicmatatjahu avatar mahakporwal02 avatar md-saif-husain avatar moritzkalwa avatar officialasishkumar avatar panwauu avatar prayutsu avatar princerajpoot20 avatar ritik307 avatar ron-debajyoti avatar rowlandbanks avatar sambhavgupta0705 avatar samridhi-98 avatar savio629 avatar shurtu-gal avatar slowikowskiarkadiusz avatar smoya avatar sumantxd avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

modelina's Issues

Enable file input

Reason/Context

If you have your input in a file you have to load your file before using the generator instead of just providing a file path and I feel like the library should be able to handle such inputs.

The files we should accept are YAML and JSON files.

Description

This will be a minor change and improve the overall interaction with the library.

Schema dependencies should be iterated

Reason/Context

Currently we do not iterate schema dependencies. We should iterate them and combine properties/items/types etc together.

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "credit_card": { "type": "number" }
  },
  "required": ["name"],
  "dependencies": {
    "credit_card": {
      "properties": {
        "billing_address": { "type": "string" }
      },
      "required": ["billing_address"]
    }
  }
}

should merge it together to give the model:

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "credit_card": { "type": "number" },
    "billing_address": { "type": "string" }
  },
  "required": ["name"]
}

Reason why billing_address is not added to the list of required properties, is that it is a runtime requirement that we cannot do anything about when rendering the model.

see more here: http://json-schema.org/understanding-json-schema/reference/object.html#schema-dependencies

Pass generator to renderers

At the moment we don't have generator (like TypeScript generator - https://github.com/asyncapi/generator-model-sdk/blob/master/src/generators/typescript/TypeScriptGenerator.ts) instance inside the renderers. This will facilitate many things related to rendering models.

Use cases: for example render enum type inside class type in Java. At the moment we must do workaround like here https://github.com/asyncapi/java-spring-template/pull/121/files#diff-812b0ef67b54ce5586eb2192840e7dbdb0c16608074f37f67a896fe733ff9436R49 to achieve inline enum in class.

Improve naming of getImmediateDependencies function

Reason/Context

We need to find a replacement name for this function since it is not clear what it does.

https://github.com/asyncapi/generator-model-sdk/blob/0a8f5e3093620a4ff064361c896fc6955ad136e0/src/models/CommonModel.ts#L186

Description

What it means is any direct/immediate/nearest dependencies of models. An example could modelA have the property prop1 that refers to another model - modelB. The immediate dependencies for modelA is then modelB because it is referenced through prop1. If modelB had any dependencies on its own, modelA would not return those since it is not the immediate dependencies.

@magicmatatjahu suggested getNearestDependencies as a replacement which I agree could be a good solution, any other suggestions?

Generate enum and use it for values.

Context

I have a function saveProject which save a asyncApiMessage as a document to a database.
This document is message wrapped in Envelope with additional properties.
A message looks like this:

export interface MessageProjectCreated { //this is what is generated from generator-model-sdk everything else is my code
   eventName: "tms.event.project.created";
};
export const message: MessageProjectCreated = {
   eventName: "tms.event.project.created",
};

My document look like this:

export enum EventName {
  ProjectCreated = "tms.event.project.created",
  ...
};
export interface Envelope<T> {
   name: EventName;
   payload: T;
};

export const document: Envelope<MessageProjectCreated> = {
   name: message.eventName,
   payload: event,
};

Reason#1

My problem with this is that typescript treat generated "tms.event.project.created" as a string not as part of EventName.
To make this work I had to replace Envelope with this.

export interface Envelope<T> {
   name: EventName | string; // a hacky way to make use of generated types
   payload: T;
};

Reason#2

I have to declare EventName enum by my own which can lead to unconsistency between event names from specification and enum written by me.

Description

I need a way to create enum in asyncapi and a way to specify field type as enum element.
So something like this: (I modified json below from this answer in SO)

{
  "$id":"MessageProjectCreated",
  "properties": {
    "eventName": {
      "$ref": "http://example.com/some_schema.json#/definitions/EventName/tms.event.project.created"
    }
  }
}
{
  "$id": "http://example.com/some_schema.json",
  "definitions": {
    "EventName": {
      "enum": [ "tms.event.project.created"]
    }
  }
}

should generate something like this.

export enum EventName {
  ProjectCreated = "tms.event.project.created",
};
export interface MessageProjectCreated {
   eventName: EventName.ProjectCreated;
};

Implement GoLang generator and renderers

Implement GoLang generator and renderers like in TS or Java.

Generator should have possibility to define presets for struct, enum and normal type type.

Additional: create presets for serialisation/converters from json.

Infer schemas from dependencies data

Given schema with dependencies data:

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "credit_card": { "type": "number" }
  },
  "required": ["name"],
  "dependencies": {
    "credit_card": {
      "properties": {
        "billing_address": { "type": "string" }
      },
      "required": ["billing_address"]
    }
  }
}

should be transformed and simplified:

{
  oneOf: [
    {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "credit_card": { "type": "number" },
        "billing_address": { "type": "string" }
      },
      "required": ["name", "credit_card", "billing_address"]
    },
    {
      "type": "object",
      "properties": {
        "name": { "type": "string" }
      },
      "required": ["name"]
    }
  ]
}

Related to #79

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.


Cannot push to the Git repository.

semantic-release cannot push the version tag to the branch master on the remote Git repository with URL https://x-access-token:[secure]@github.com/asyncapi/generator-model-sdk.git.

This can be caused by:


Good luck with your project ✨

Your semantic-release bot 📦🚀

Interfaces are not being documented via JSdoc

Describe the bug

Our current flow for generating docs is first transpile TypeScript to Javascript, then run jsdoc2md to generate the final Markdown file. See https://github.com/asyncapi/generator-model-sdk/blob/master/package.json#L62.

Documenting interfaces in jsdoc is totally possible thanks to the @interface tag. However, the tag is not supported by TS compiler. See https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html.

That means we cannot generate documentation for interfaces later on with jsdoc2md as those tags are not added ever.

Expected behavior

We somehow generate documentation for interfaces.

I would suggest we re-evaluate the current flow so we can either:

  1. Generate the docs directly from TS:
  2. Rather we transpile using an alternative like https://github.com/SoftwareBrothers/better-docs#how-it-works

Remove Eslint warnings

Reason/Context

Currently we ignore warnings but it needs to be addressed. Especially the complexity check. After that we need to force no warnings.

getImmediateDependencies assumes items cannot be an array.

Describe the bug

Since CommonModel items can be an array the function should handle that and not assume items are only one type.

How to Reproduce

Location of function: https://github.com/asyncapi/generator-model-sdk/blob/9dec1e58ec5da21e8dbd80fc44075c53f156b830/src/models/CommonModel.ts#L232
Remember to change the tests as well.

Caught by @magicmatatjahu here: #183 (comment)

Expected behavior

Should handle CommonModel | CommonModel[] and return accurately.

Fix simplification of tuple

Describe the bug

When I fixing rendering the tuple for TS, I found the bug.

At the moment given schema:

{ "type": "array", "items": [{ "type": "string" }, { "type": "number" }] }

produces given CommonModel:

CommonModel {
  originalSchema: Schema { type: 'array', items: [ [Schema], [Schema] ] },
  type: [ 'string', 'number' ]
}

and this is not true, because it's a tuple, not an array of unions. We should fix it.

Rendered models should be compiled as a blackbox test

Reason/Context

At the moment we do not directly test the correctness of the output. So when we receive PRs changing the rendering in any way we have no way of knowing whether the output can actually compile.

We already encountered this issue and having something in place to handle this logic would be awesome.

Description

I would suggest we implement a GitHub action per language we output to. For now, this means we would create a workflow for Java, Typescript, and JS.

Questions that need to be answered:

  • Should this workflow run for each PR?
  • Should we create one global input file which covers all possible values the output can be?
  • Say we scale to 10 languages, how much time would that affect the "wait time" for checks to complete?
  • What about different versions of languages, should that be tested?

Ensure we are JSON Schema draft 7 compliant

Reason/Context

We need to ensure that this library is 100% JSON Schema draft 7 compliant since this is the main input source of the library.

Description

We need to use their test compatibility suite. I suggest we wrap any of their schemas in an object property to force the library to generate models.

Something similar to:

{
    type: "object",
    properties: {
        "testProp": <suite test schema>
    }
}

I am not sure how we can ensure the correct output though🤔 Any ideas?

Figure out how we can introduce rendering of inheritance

Reason/Context

One of the strong features of the CommonModel is the possibility to use extend which contains a list of $ids of the CommonModels that the current model should extend. Therefore we need to figure out how we render this in the different languages

Wrong check in regard to whether a CommonModel should be extended

Describe the issue

As the comment states here:
https://github.com/asyncapi/generator-model-sdk/blob/cc42b3ae7753044c173cbdd16cfa4f9fb7cb2c34/src/simplification/Utils.ts#L7

We need to refactor a bit to better determine if a CommonModel is a simple model, generally whenever the model type != object, or an independent model which should be generated as such.

The reason why we cannot use the new check yet is that this function is used before the properties extend and properties are even sat on the model.

Also reflect on whether we can do modify the check to work with the current implementation without having to refactor.

JSON example preset

Reason/Context

I have had a few use-cases now where I need to generate example JSON based on JSON Schema, and if we can generate the underlying model we should be able to generate an example JSON for that model as well.

Description

It does not have to be fancy just a quick and dirty example will suffice.

The direction of the (default) presets

While integrating this library into ts-nats-template I got frustrated because my expected outcome was not aligned with the default preset. And this comes down to my basic understanding of it was wrong, the setup is great because it offers many different ways of rendering a model. However, I realized that we need a clear direction in which way we want to take these presets.

As I see it we have 3(+) choices in which our default preset could move towards:

  1. The complete model
  2. The data transfer model
  3. The minimalistic
  4. Do you have another suggestion?

The complete model

This model gives you everything in regards to rendering a complete model. Where you have access to not only the properties but additionalProperties, patternProperties, inheritance, etc.

The data transfer model

Optimized for serializing and deserializing data.

The minimalistic

Just the bare minimum properties, basically what we have now.

Questions

  • What should the default preset be?
  • What presets should we include with this library by default?
    • How many different presets should the library provide? I.e. all 3 before-mentioned presets?

Related issues and PRs

All the following issues/PRs are blocked until we figure out the direction.

The following issues/PRs are somewhat blocked since this could trigger a rewrite unless we are careful.

Create a CLI

Reason/Context

Because this library should not be directly associated with the generator but a separate library we should introduce a CLI for the library.

Description

Be aware that this CLI might be part of a larger CLI and should be implemented with extensibility in mind.

Related issues

#132

language specific keywords are rendered as property names

Describe the bug

There are certain keywords that are reserved. This is different between languages, we need to make sure we dont render such properties as-is.

How to Reproduce

For example, in Java, you cannot have a property named enum.

Expected behavior

I expected the library to handle any property names.

Enable TS input

Reason/Context

It would be awesome to take TS code and be able to output models in any other language.

Description

There already exist libraries we can leverage to minimize the effort to integrate this where we could use the already existing JSON Schema process or create a custom one.

Link shared by Kevin Swiber: https://github.com/YousefED/typescript-json-schema

Enum isn't render properly in as TS class property

If we have property as enum like:

state:          { type: "string", enum: ["Texas", "Alabama", "California", "other"] }

then we have rendered in TS:

state: string

but should be:

state: "Texas" | "Alabama" | "California" | "other"

Document how to use the sdk

We should document how user/dev should use our model-sdk in templates - here we should have examples with React and Nunjucks templating - in library, or in normal app.

Documents should contain also complex cases like:

  • how to use pass options to generators.
  • how to use presets in given language - with overriding case, concatenation etc.

Also we should update main Readme.md to provide information what user/dev should do to use out package and small introduction to package like in react-sdk package.

Rewrite whole tests related to simplifiers and processors

As in title. At the moment, if someone change something in simplifiers and processors must update (sometimes) 20-30 test cases. It wouldn't be a problem, but the tests are written as integration tests that each test that tests the results of a function with content stored in separate files. Some files are big enough that it is tiresome. Each test should be an individual test that tests only one part of the scheme, not the whole.

Add enum for asyncapi message $id`s

Reason/Context

I created generator which generate types and Joi validators.
I generate validator with code like below.

validators.enjoiOne("MessageJobCreated")

As you can see I must pass string what is a little bit dirty solution.
This string is $id of message from asyncapi yaml file but there is no way to access it from generated types.

Description

I would like to have access to enum with all messages $id`s as key and value.
I guess it may look similar to this one.

export enum AsyncApiMessages {
   MessageJobCreated = "MessageJobCreated" 
}

Add a validator infront of the input processor

Describe the bug

We are expecting the input JSON Schema draft 7 to be a document that can validate correctly, but we do nothing when this is not the case.

Example: If a schema uses combination schemas that each contain the same property name, it will try to merge the two common models. However, if it is from an allOf combination schema, it should "fail" somehow if conflicts occur. Since the input JSON Schema could never validate any data we should consider it an invalid input.

How to Reproduce

Given the JSON Schema:

{
  "allOf": [
    {
      "type": "object",
      "properties": {
        "somePropName": {
          "type": "number"
        }
      }
    },
    {
      "type": "object",
      "properties": {
        "somePropName": {
          "enum": [
            "residential",
            "business"
          ]
        }
      },
      "required": [
        "type"
      ]
    }
  ]
}

It would create the simplified model:

{
  "type": "object",
  "properties": {
    "somePropName": {
      "type": [
        "number",
        "string"
      ]
    }
  }
}

Which is incorrect. However, it is the input that is incorrect, since the JSON Schema can never validate the input.

Expected behavior

I would expect the model generator to fail the generation, or handle invalid inputs somehow.

Implement C# generator and renderers.

Implement C# generator and renderers like in TS or Java.

Generator should have possibility to define presets for class, enum type.

Additional: create presets for serialisation/converters from json.

Library entirely ignore required field in `components/schemas` while merging required fields

Describe the bug

Library entirely ignore required field in components/schemas while merging required fields.
Probably schemas are not instanced to CommonModel 😢 😞

How to Reproduce

Run parser with those components and check if project interface has id?: string.

components:
  schemas:
    EventEnvelope:
      $id: EventEnvelope
      description: "Envelope shared between messages"
      type: object
      required:
        - eventName
      properties:
        eventName:
          type: string
    
    Project:
      $id: Project
      type: object
      required: [id]
      properties:
        id:
          type: string

  messages:
       someEventName:
        $id: someEventName
        allOf:
          - $ref: "#/components/schemas/EventEnvelope"
          - type: object
            properties:
              data: object
              properties: 
                project: 
                  $ref: "#/components/schemas/Project"
  • Screenshots
  • Link to GitHub repository with project that has issues
  • Files that can help reproduce the issue, like your AsyncAPI file, either paste inside the issue in a code block or in a sharable gist

Expected behavior

project interface has id: string

Workaround

Set required fields everytime you do $ref: "#/components/schemas/Project". It has to be done everywhere you do reference, if not field will not be required.

Improve typings in project

We should improve typings in whole project to improve DX:

  • improve generic types in AbstractGenerator, AbstractRenderer - #60
  • improve generic types in presets: CommonPresets... and in Preset abstract interface to include typings for options for given preset.

Simplify metadata fields in combined schemas

We don't support simplification for metadata fields like minItems, minimum, maxLength etc. in combined schemas like anyOf, oneOf, allOf. This issue is a tracker for each missed field.

  • Number/Integer:
    • minimum
    • exclusiveMinimum
    • maximum
    • exclusiveMaximum
    • multipleOf
  • String:
    • minLength
    • maxLength
    • pattern
  • Array:
    • minItems
    • maxItems
    • uniqueItems
  • Object:
    • minProperties
    • maxProperties
  • All
    • examples

Please update if something was not included.

Naming convention for type and property

We should have naming convention for models, it means naming style for model type (included refs) or property. I suggest to have in default option of generator namingConvention object with type and property fields. At the moment developer must change manually name of type or property, e.g. type is called some_class and in Java should be pascalCased - SomeClass - at the moment we must do it inside code, by helpers - finally user of data-model generation cannot change naming style, because it is hardcoded.

My suggestion for signature:

options = {
   // rest of default options
   namingConvention: {
     type: (name: string, model: CommonModel) => string,
     property: (name: string, model: CommonModel) => string,
   }
}

Using in code (preset):

const preset = {
  class: {
    property({ renderer, propertyName, property }) {
      const name = renderer.nameProperty(propertyName, property);
      // logic
    }
  }
}

Bidirectional preset flow

At the moment we have only one way flow in presets. It means that render starts from defaultPreset, goes through every custom preset and return to output final string. Sometimes we need to adjust to final part of model something after rendering the rest of custom presets:

e.g. we have CUSTOM_PRESET_1, CUSTOM_PRESET_2, in CUSTOM_PRESET_1 I wanna run first CUSTOM_PRESET_2 and then add something to final string by CUSTOM_PRESET_1. Of course I can change order of CUSTOM_PRESET_2 with CUSTOM_PRESET_1 but some actions cannot be achieved by changing order - we wanna render some property, go through by every preset for property, but in final string, render something additional above the property... in this case we have problems:

Of course bidirectional preset flow doesn't fix all problem related to overwriting presets result - some preset must be render as last - but we can avoid some problems.

Maybe bidirectional preset flow isn't a good thing, how do you think about that? Maybe another suggestion to resolve above problems?

should render additionalProperties

Reason/Context

Currently the CommonModel does include additionalProperties, however they are not rendered as properties in the generator

Questions:

  • How should we handle additionalProperties if we already have a property called additionalProperties?

AsyncAPI schemas should use their id not ours

Describe the bug

Currently, in templates, all you have access to is the uid() function from the AsyncAPI parser for the schema which returns a different name than what the model is generated with. This should not be the case.

Why? Because I cannot change what I have access to, we should not interpret another id if one is already given by another tool.

Find a proper name for the library

generator-model-sdk can be confusing for someone in AsyncAPI community and also for people outside the community. It doesn't need the generator to work, it's a standalone library like QuickType, so we should change name of the repo and project. Any ideas?

My proposition is Modelina. It is a polish word for modelling clay. There is possible to form something (models) out of modeling clay (input data) - like in programming 🤣 . And the model is also in the name :)

Document how to implement generator for language

After first release, we will have only 3 languages which we will support - JS, TS and Java with custom presets, e.g. with Jackson support (Java). We should document how community can contribute to our model-sdk with implementation of new languages and presets. Documents should contain:

  • how to extend AbstractGenerator.
  • how to extend AbstractRenderer.
  • how to write custom renderers and how to connect them to the generator.
  • how to write default preset
  • how to write custom presets for given language

Also mentioned in #70

  • how generator works

Add support for JSON Schema files as input

Reason/Context

Considering that we are potentially gonna have parsers written in other languages (not just JS. E.g. https://github.com/asyncapi/parser-go), It makes sense to support an input type that can be used to load schemas generated by any of those.

This will allow using any convenient parser, no matters the language they are implemented on.

This feature request was motivated by this PR review comment.

Description

Ideally, the parsers should be able to generate JSON Schema files so they can be used later by this generator.

Add logging interface

Reason/Context

We should provide a logging interface for users of the library to get notified on internal logs.

Use-case:

  • In case two simplified models need to be merged some properties such as $id are overwritten, in these special cases I would like to inform the user that a clash between properties occurred and the output might not contain the expected output.
  • As a user I would like to see which models are generated

Description

I suggest we provide a logging interface so the users of the library can decide which logs they would like to do something with.

This could for example include 4 levels:

  • Debug
  • Info
  • Warning
  • Error

Iterate pattern properties

Reason/Context

Pattern properties should be merged with additional properties
Pattern properties should be included the same way that additionalProperties are. Meaning it should be moved from Schema to CommonSchema.

Renderer does not render extended classes which is simplified by default

By default, we simplify JSON schema into multiple classes such that:

{
  "allOf": [
    {
      "type": "object",
      "properties": {
        "street_address": {
          "type": "string"
        },
        "country": {
          "type": "string"
        }
      }
    },
    {
      "type": "object",
      "properties": {
        "street_address2": {
          "type": "string"
        },
        "country2": {
          "type": "string"
        }
      }
    }
  ]
}

should render 3 classes, root which extends rootAllOf1 and rootAllOf2, and then the later two classes are then separated. But since the renderer is not ready for this we need to switch to defaulting to merging the allOf schemas instead. So no schemas are left behind.

Typescript renders encapsulation for non-required properties wrong

Describe the bug

non-required properties renderers a wrong signature for encapsulation.

How to Reproduce

Given the following JSON schema

{
	"type": "object",
	"$id": "GeneralReply",
	"properties": {
		"status_code": {
			"type": "integer"
		},
		"status_message": {
			"type": "string"
		}
	}
}

It is rendered to

export class GeneralReply {
  private _statusCode?: number;
  private _statusMessage?: string;

  constructor(input: {
    statusCode?: number,
    statusMessage?: string,
  }) {
    this._statusCode = input.statusCode;
    this._statusMessage = input.statusMessage;
  }

  get statusCode(): number { return this._statusCode; }
  set statusCode(statusCode: number) { this._statusCode = statusCode; }

  get statusMessage(): string { return this._statusMessage; }
  set statusMessage(statusMessage: string) { this._statusMessage = statusMessage; }
}

It is however unable to compile based on the following error messages:

> tsc

src/messages/GeneralReply.ts:13:30 - error TS2322: Type 'number | undefined' is not assignable to type 'number'.
  Type 'undefined' is not assignable to type 'number'.

13   get statusCode(): number { return this._statusCode; }
                                ~~~~~~~~~~~~~~~~~~~~~~~~

src/messages/GeneralReply.ts:16:33 - error TS2322: Type 'string | undefined' is not assignable to type 'string'.
  Type 'undefined' is not assignable to type 'string'.

16   get statusMessage(): string { return this._statusMessage; }
                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~


Found 2 errors.

Implement Python generator and renderers

Implement Python generator and renderers like in TS or Java.

Generator should have possibility to define presets for class, enum type.

Additional: create presets for serialisation/converters from json.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.