Giter VIP home page Giter VIP logo

protobuf-schema-parser's Introduction

ProtoBuff Data Types Schema Parser

A schema parser for Protocol Buffers data types. For ProtoBuff 2 and 3 schemas.

There is no explicit distinction between ProtoBuff 2 and 3. You dont have to expect any errors if your schemaFormat is application/vnd.google.protobuf;version=2 defined, but your schema is proto3.

Version >= 2.0.0 of package is only supported by @asyncapi/parser version >= 2.0.0.

This package is browser-compatible.

Installation

npm install @asyncapi/protobuf-schema-parser
// OR
yarn add @asyncapi/protobuf-schema-parser

Usage

import {Parser} from '@asyncapi/parser';
import {ProtoSchemaParser} from '@asyncapi/protobuf-schema-parser'

const parser = new Parser();
parser.registerSchemaParser(ProtoSchemaParser());

const asyncapiWithProto = `
asyncapi: 2.0.0
info:
  title: Example with ProtoBuff
  version: 0.1.0
channels:
  example:
    publish:
      message:
        schemaFormat: 'application/vnd.google.protobuf;version=3'
        payload: |
            message Point {
                required int32 x = 1;
                required int32 y = 2;
                optional string label = 3;
            }

            message Line {
                required Point start = 1;
                required Point end = 2;
                optional string label = 3;
            }
`

const {document} = await parser.parse(asyncapiWithProto);
const {Parser} = require('@asyncapi/parser');
const {ProtoSchemaParser} = require('@asyncapi/protobuf-schema-parser');

const parser = new Parser();
parser.registerSchemaParser(ProtoSchemaParser());

const asyncapiWithProto = `
asyncapi: 2.0.0
info:
  title: Example with ProtoBuff
  version: 0.1.0
channels:
  example:
    publish:
      message:
        schemaFormat: 'application/vnd.google.protobuf;version=3'
        payload: |
            message Point {
                required int32 x = 1;
                required int32 y = 2;
                optional string label = 3;
            }

            message Line {
                required Point start = 1;
                required Point end = 2;
                optional string label = 3;
            }
`

const {document} = await parser.parse(asyncapiWithProto);

Place your protoBuff schema as string in payload to get it parsed.

References are NOT supported:

  • no support for $ref
  • no support for import, except the default google types:
    • google/protobuf/*
    • google/type/*

Comments

Each field of a message may have a comment witch will be reflected as json schema description. Furthermore, the comment can contain the following annotations:

message Point {
    /*
     * The cordinate on the x axis.
     * @Default 99
     * @Min 0
     * @Max 100 
     */
    required int32 x = 1;
    
    /*
     * The cordinate on the y axis.
     * @Default 12
     * @Min 0
     * @Max 100 
     */
    required int32 y = 2;
    optional string label = 3;
}

Per field annotation

annotation description
@Example json schema examples keyword. Can exists multiple times. If used with an complex type, an single lines json object hast to be used.
@Min or @Minimum json schema numeric validator
@Max or @Maximum json schema numeric validator
@Pattern json scheme string validator
@ExclusiveMinimum json schema numeric validator
@ExclusiveMaximum json schema numeric validator
@MultipleOf json schema numeric validator
@MinLength json scheme string validator
@MaxLength json scheme string validator
@MinItems json scheme array validator
@MaxItems json scheme array validator
@Default json schema default value

Per message annotation

annotation description
@RootNode If there are multiple types without an parent you can give a hint to the root node with this annotation.

protobuf-schema-parser's People

Contributors

asyncapi-bot avatar derberg avatar greenrover avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

derberg dermolly

protobuf-schema-parser's Issues

Recursive protobuf schema causes stackoverflow

Describe the bug

Recursive messages cause stackoverflow.

How to Reproduce

Define a protobuf schema with a recursion, e.g.

asyncapi: 3.0.0
info:
  title: Account Service
  version: 1.0.0
  description: This service is in charge of processing user signups
channels:
  userSignedup:
    address: user/signedup
    messages:
      UserSignedUp:
        $ref: '#/components/messages/UserSignedUp'
operations:
  sendUserSignedup:
    action: send
    channel:
      $ref: '#/channels/userSignedup'
    messages:
      - $ref: '#/channels/userSignedup/messages/UserSignedUp'
components:
  messages:
    UserSignedUp:
      payload:
        schemaFormat: application/vnd.google.protobuf;version=3
        schema: |
          syntax = "proto3";
          package Example;

          message A {
            string something = 1;
            repeated A recursive = 2;
          }

It will throw a Error thrown during AsyncAPI document parsing. Name: RangeError, message: Maximum call stack size exceeded, stack: RangeError: Maximum call stack size exceeded\n at Type.lookupTypeOrEnum

Expected behavior

Don't break. Even if it means recursive functions are not displayed properly in the generated docs, but it shouldn't make an asyncapi definition unusable, because certain things require such recursive protobuf messages

Support ref and import

Reason/Context

I'll just describe my use case:

We have a fairly large set of protobuf definitions which:

  • Are hosted in a central location (so ref support is useful)
  • Use 'import' to factor out common sub-types to avoid code duplication (so import support is useful)

Our company is large enough that contracts are often multi-party, e.g. some protobufs come from one system but get encoded as base64 and put in a json envelope by another service, etc. When defining contracts we often want to describe the parts we're responsible for, but provide references to the schemas that aren't ours.

Description

  • What changes have to be introduced?

I'm not quite sure.

  • Will this be a breaking change?

I would not think so because this adds functionality.

  • How could it be implemented/designed?

I'm not sure at this time.

Consider upgrading the code to be supported by AsyncAPI Parser-JS v2

Reason/Context

Parser-JS 2.0 (released as rc at this moment) is a completely new parser, with a lot of changes. The bigger ones are:

  • Completely rewritten to TS.
  • Implements the new parser-api, in particular, the version found in this PR.
  • Uses Spectral as parser/linter with the AsyncAPI ruleset.
  • Lots of other improvements, especially around Developer Experience.

Considering that Spectral now validates the documents before parsing, we had to update the interface for Schema parsers to separate validation from parsing. This change also introduced better errors and the way we handle them. (see this issue and related).

Description

Note that Parser-JS v2 uses TypeScript so feel free to use it as well.

Need for urgent changes in GitHub Actions automation

This issue defines a list of tasks that need to be performed in this repo to make sure it's ci/cd automation works long term without any issues.

It is up to maintainers to decide if it must be addressed in one or multiple PRs.

Below are 3 different sections describing 3 different important ci/cd changes.

IMPORTANT-START
For GitHub workflows that contain This workflow is centrally managed in https://github.com/asyncapi/.github/ you do not have to perform any work. These workflows were already updated through the update in .github. The only exception is the workflows related to nodejs release. More details in Upgrade Release pipeline - in case of nodejs projects section
IMPORTANT-END

Deprecation of way data is shared between steps

Every single GitHub Action workflow that has echo "::set-output name={name}::{value}" need to be updated to follow echo "{name}={value}" >> $GITHUB_OUTPUT

We do not yet know when set-output will stop working. Previous disable date was 31.05 but now then say community needs more time.

For more details read official article from GitHub

Deprecation of node12

2nd bullet point is still relevant for you even if your projects in not nodejs project

  • Every single workflow that uses setup-node action needs an update to follow v3 version of this action, and make sure minimum node 14 is used
  • Now this part is more complex. Problem with node12 is that node-based GitHub Actions were using it in majority as a runtime environment. Look for example at this action.yaml file for setup-node action v2. So the job that you have to do is go through all the workflows, and verify every single action that you use, make sure you are using the latest version that is not based on node12. I already did review a lot of actions as part of this PR so maybe you will find some actions there and can copy from me. For example action/checkout needs to be updated to v3.

Node12 end of support in action is probably September 27th.

For more details read official article from GitHub

Upgrade Release pipeline - in case of nodejs projects

ignore this section if your project is not nodejs project

You have 2 options. You can:

A. choose to switch to new release pipeline using instruction from asyncapi/.github#205

B. stay with old release pipeline, and manually update GitHub workflows and actions used in it, you can inspire a lot from this PR asyncapi/.github#226

I definitely recommend going with A

Workflows related to release:

  • .github/workflows/if-nodejs-release.yml
  • .github/workflows/if-nodejs-version-bump.yml
  • .github/workflows/bump.yml

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.