Giter VIP home page Giter VIP logo

protorefine's Introduction

protorefine

BACKGROUND

Assume we have a big project, it has many sub projects, each sub project may use protobuf types defined in other sub projects, the easiest way is put all proto files belong to different sub projects into same directory and just name it with different filename, then compile all proto files into one package. e,g:

proto/
    ...
    order.proto
    product.proto
    ...

will generate souce codes in autogen/pb directory

autogen/
    pb/
        ...
        order.pb.go
        product.pb.go
        ...

But the drawback is the final built binary file is too large, it will include all protobuf types defined in all proto files, which is not necessary.

INTRODUCTION

Fortunately, this tool is developed to find all protobuf types referenced in source codes, then extract the protobuf type definitions from center proto files repository, at last generate new proto source file which only contains protobuf types we needed.

INSTALL

go install github.com/rfancn/protorefine

USAGE

  1. it find all <pb-import-path> protobuf types referenced in source codes in <project-dir>
  2. it extract corresponding protobuf type definitions from center proto files repository in <proto-dir>
  3. it generate a new proto file <project>.proto in <output-dir> which contains all protobuf types we needed
  4. it match and copy dependent proto files from <proto-dir> to <output-dir> use import rules, import rules read from config file specified by --config option, if no --config option specified, it uses default import rules

if --output-dir not set, it will generate a random directory

EXAMPLE

  • find all project/autogen/pb protobuf types referenced in /tmp/project, extract corresponding protobuf type definitions in /tmp/proto, at last generate a new proto file project.proto in temporary directory

    protorefine --project-dir=/tmp/project --proto-dir=/tmp/proto --pb-import-path project/autogen/pb
    
  • find all project/autogen/pb protobuf types referenced in /tmp/project exclude /tmp/project/autogen directory, extract corresponding protobuf type definitions in /tmp/proto, at last generate a new proto file project.proto in /tmp/project/autogen/proto

    protorefine --project-dir=/tmp/project --proto-dir=/tmp/proto --pb-import-path project/autogen/pb --output-dir=/tmp/project/autogen/proto --skip-dirs=autogen
    
  • find all project/autogen/pb protobuf types referenced in /tmp/project, extract corresponding protobuf type definitions in /tmp/proto, generate a new proto file project.proto in /tmp/project/autogen/proto, copy dependent proto files match the rule defined in /tmp/config.toml

    protorefine --project-dir=/tmp/project --proto-dir=/tmp/proto --pb-import-path project/autogen/pb --output-dir=/tmp/project/autogen/proto --config /tmp/config.toml
    

Config

  • match: defines regular expression to match protobuf type definition content
  • file: defines the file name directly imported in generated proto file
  • dependents: defines the dependent proto files the new proto file will import indirectly

Default import rules config

[import]
    [[import.rules]]
        match="gogoproto\\.+"
        file="gogo/protobuf/gogoproto/gogo.proto"
        dependents= ["google/protobuf/descriptor.proto"]
    [[import.rules]]
        match="hdsdk.protobuf\\.+"
        file="hdsdk/protobuf/sdk.proto"
    [[import.rules]]
        match="google.protobuf.Any"
        file="google/protobuf/any.proto"

The \ character collies with toml syntax, you need to escape it with \\

protorefine's People

Contributors

rfancn avatar

Stargazers

hdget avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.