Giter VIP home page Giter VIP logo

toml-tools's Introduction

Release LICENSE code style: prettier

TOML-Tools

Introduction

This mono-repo contains a set of tools for working with the Toml configuration file format.

Why

The common Toml related tooling are Toml -> Json compilers. But compiling to Json is just one scenario of many, lets consider some other potential use cases:

  • Building a Toml Syntax Highlighter.
  • Converting Toml to Yaml.
  • Building a Toml Beautifier.
  • Validating Toml versus a given schema.
  • Providing content assist in a Toml file.

These scenarios cannot be implemented (well) using most existing Toml to Json compilers, for example:

  • Lost comments information.
    • How can we beautify or convert Toml to Yaml if we don't possess the comments information?
  • Lack of Token type & positioning information.
    • How could we syntax highlight(paint) parts of a Toml file if we do not know where every single Token starts and ends (Commas/Parenthesis/Literals/...)?
  • Failing on the first error
    • How would we implement content assist on a Toml file currently being edited if our parser fails on the first error?
  • Lack of full position information
    • How will we provide useful Schema validation errors if we lack full position information?
    • Full position information would also be needed for the content assist scenario.

How

This mono repo will contain two types of packages.

  • Low level Infrastructure (Toml Lexer & Parser implemented using Chevrotain) with advanced capabilities to support the complex scenarios described above.
  • Toml (end user) Tooling (Beautifier/Compiler/Schema Validator/...) that would be implemented using the advanced low level infrastructure packages mentioned above.

Status (Alpha)

Infrastructure packages:

Tooling Packages:

toml-tools's People

Contributors

bd82 avatar coliff avatar evolutics avatar jaysonsantos avatar jounqin avatar weirongxu avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

weirongxu freight-trust jasikpark jaysonsantos professormanhattan duongtd1998 voltiso nextnet-works pedomaster

toml-tools's Issues

Constantly crashes with "Cannot read properties of undefined (reading 'tokenTypeIdx')"

Adding prettier-plugin-toml to an existing project using Prettier 3 sees it constantly crash with Cannot read properties of undefined (reading 'tokenTypeIdx'). It seems to have when Prettier is processing any file (not just TOML files).

This seems to be as much debugging info as I can get:

$ prettier --ignore-unknown --check '**' --log-level debug

[debug] normalized argv: {"":["**"],"cache":false,"check":true,"color":true,"editorconfig":true,"ignoreUnknown":true,"logLevel":"debug","configPrecedence":"cli-override","debugRepeat":0,"ignorePath":[".gitignore",".prettierignore"],"plugins":[],"_":["**"],"__raw":{"_":["**"],"cache":false,"check":true,"color":true,"editorconfig":true,"ignore-unknown":true,"log-level":"debug","config-precedence":"cli-override","debug-repeat":0,"ignore-path":[".gitignore",".prettierignore"],"plugin":[]}}
Checking formatting...
[debug] resolve config from '[some-file-in-my-project]'
[debug] loaded options `[contents-of-my-.prettierrc-file]`
[error] Cannot read properties of undefined (reading 'tokenTypeIdx')

I've tried removing all my Prettier config, running it on a single TOML file, running it on a single non-TOML file etc., but it constantly crashes when the plugin is enabled (either in the config file or on the command line).

This might be the same problem as in #42.

Readme

  • Root readme.
  • Lexer readme.
  • Parser readme.
  • npm Badages

prettier-plugin-toml removes blank lines after comments

Example:

# This is a TOML comment

# This is a multiline
# TOML comment

str1 = "I'm a string."

Expected: code should not be changed.

Actual:

# This is a TOML comment
# This is a multiline
# TOML comment
str1 = "I'm a string."

Looking for new maintainer

I no longer have time to maintain this project.
If anyone is interested in taking over this project please reply in this thread.

Toml Ambiguities

  • Float in key

    [95.22.looks.like.a.float]
name = "John"

  • "[[" vs "[" (Table of Arrays vs Array vs Table vs array with first element another array)

       [[Table.of.arrays]]
   # Array value with first item being a nested array
   items = [[1,2,3], [4,5,6]]

Benchmark

Pure node.js is sufficient for the initial version

Yarn PnP doesn't work

in order for yarn berry pnp to work correctly dependencies must be fully declared

[error] prettier-plugin-toml tried to access @toml-tools/lexer, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.
[error] 
[error] Required package: @toml-tools/lexer
[error] Required by: prettier-plugin-toml@npm:0.3.1 (via /Users/nqy642/IdeaProjects/iw-pm-backend/.yarn/cache/prettier-plugin-toml-npm-0.3.1-a4a923f5b6-9c34cc4a35.zip/node_modules/prettier-plugin-toml/lib/)
[error] 
[error] Require stack:
[error] - /Users/nqy642/IdeaProjects/iw-pm-backend/.yarn/cache/prettier-plugin-toml-npm-0.3.1-a4a923f5b6-9c34cc4a35.zip/node_modules/prettier-plugin-toml/lib/printer.js
[error] - /Users/nqy642/IdeaProjects/iw-pm-backend/.yarn/cache/prettier-plugin-toml-npm-0.3.1-a4a923f5b6-9c34cc4a35.zip/node_modules/prettier-plugin-toml/lib/api.js
[error] - /Users/nqy642/IdeaProjects/iw-pm-backend/.yarn/cache/prettier-npm-2.7.1-d1f40f5e1a-55a4409182.zip/node_modules/prettier/index.js
[error] - /Users/nqy642/IdeaProjects/iw-pm-backend/.yarn/cache/prettier-npm-2.7.1-d1f40f5e1a-55a4409182.zip/node_modules/prettier/cli.js
[error] - /Users/nqy642/IdeaProjects/iw-pm-backend/.yarn/cache/prettier-npm-2.7.1-d1f40f5e1a-55a4409182.zip/node_modules/prettier/bin-prettier.js

prettier-plugin-toml Feature Request: Magic Trailing Comma

In Python I got used to the Magic Trailing Comma which prevents collapsing a list into one line, if there is a trailing comma in the last element.

There are cases where you put a short collection or function call in your code but you anticipate it will grow in the future.

Input

[example]
foo = [
    "bar",
    "baz",
]
hello = [
    "world"
]

Expected output

[example]
foo = [
    "bar",
    "baz",
]
hello = ["world"]

Sorting support

As a formatter an optional sorting feature would be quite appreciated as it would help keeping config files in sync and easy to compare across repos. Same thing exists for ini files.

Generate Visitor TypeScript APIs

These are currently manually maintained which will lead to inconsistencies in the future.
Perhaps this should be handled by Chevrotain itself.
See: Chevrotain/chevrotain#851

Or maybe a simple solution could be built in this repo first and later made more generic
for the general library use case.

Lexer newline handling

Ensure the default newline handling matches that of the TOML spec
or customize it otherwise...

release prettier-plugin-toml 0.3.2 to npmjs

Could you please release latest code for prettier-plugin-toml to npmjs.
Currently 0.3.1 having this error:

 TypeError: Cannot read property 'tokenTypeIdx' of undefined
    at /project/node_modules/@toml-tools/parser/node_modules/chevrotain/lib/src/scan/tokens.js:91:78
    at Object.forEach (/project/node_modules/@toml-tools/parser/node_modules/chevrotain/lib/src/utils/utils.js:85:30)
    at /project/node_modules/@toml-tools/parser/node_modules/chevrotain/lib/src/scan/tokens.js:90:17
    at Object.forEach (/project/node_modules/@toml-tools/parser/node_modules/chevrotain/lib/src/utils/utils.js:77:30)
    at assignCategoriesTokensProp (/project/node_modules/@toml-tools/parser/node_modules/chevrotain/lib/src/scan/tokens.js:87:13)
    at Object.augmentTokenTypes (/project/node_modules/@toml-tools/parser/node_modules/chevrotain/lib/src/scan/tokens.js:32:5)
    at TomlParser.RecognizerEngine.initRecognizerEngine (/project/node_modules/@toml-tools/parser/node_modules/chevrotain/lib/src/parse/parser/traits/recognizer_engine.js:86:18)
    at new Parser (/project/node_modules/@toml-tools/parser/node_modules/chevrotain/lib/src/parse/parser/parser.js:69:14)
    at new TomlParser (/project/node_modules/@toml-tools/parser/lib/parser.js:6:5)
    at Object.<anonymous> (/project/node_modules/@toml-tools/parser/lib/api.js:4:16)
    at Module._compile (node:internal/modules/cjs/loader:1091:14)
    at Module._compile (/usr/local/lib/node_modules/ts-node-dev/node_modules/source-map-support/source-map-support.js:547:25)
    at Module._extensions..js (node:internal/modules/cjs/loader:1120:10)
    at require.extensions..jsx.require.extensions..js (/private/var/folders/cx/hqb2r_t50q7761ls_dp99qzm00011z/T/ts-node-dev-hook-6387655258481435.js:114:20)
    at Object.nodeDevHook [as .js] (/usr/local/lib/node_modules/ts-node-dev/lib/hook.js:63:13)
    at Module.load (node:internal/modules/cjs/loader:971:32)
    at Function.Module._load (node:internal/modules/cjs/loader:812:14)
    at Module.require (node:internal/modules/cjs/loader:995:19)
    at require (node:internal/modules/cjs/helpers:92:18)
    at Object.<anonymous> (/project/node_modules/prettier-plugin-toml/lib/api.js:2:19)
    at Module._compile (node:internal/modules/cjs/loader:1091:14)
    at Module._compile (/usr/local/lib/node_modules/ts-node-dev/node_modules/source-map-support/source-map-support.js:547:25)
    at Module._extensions..js (node:internal/modules/cjs/loader:1120:10)
    at require.extensions..jsx.require.extensions..js (/private/var/folders/cx/hqb2r_t50q7761ls_dp99qzm00011z/T/ts-node-dev-hook-6387655258481435.js:114:20)
    at Object.nodeDevHook [as .js] (/usr/local/lib/node_modules/ts-node-dev/lib/hook.js:63:13)
    at Module.load (node:internal/modules/cjs/loader:971:32)
    at Function.Module._load (node:internal/modules/cjs/loader:812:14)
    at Module.require (node:internal/modules/cjs/loader:995:19)
    at require (node:internal/modules/cjs/helpers:92:18)
    at /project/node_modules/prettier/index.js:57503:6
    at Array.map (<anonymous>)
    at Object.load (/project/node_modules/prettier/index.js:57501:118)
    at Object.load [as loadPlugins] (/project/node_modules/prettier/index.js:16612:23)
    at /project/node_modules/prettier/index.js:57540:28
    at Object.format (/project/node_modules/prettier/index.js:57562:12)

Thanks in advanced

Compiler to Json package (@toml-tools/toml-to-json)

  • Evaluate performance advantages of not using the CST for this.
  • Evaluate performance advantages of using custom token patterns, could those be used
    to avoid post-processing?
    Possible use cases:
    - when lexing a date could we create the JS Date Object on the fly?
    - processing backslash line terminator in multiline strings
    - stripping strings delimiters
    - Numerical values conversion to Number during lexing.
  • Evaluate additional "fastest" API relevance (e.g only offset position tracking).

`prettier-plugin-toml` does not seem to work using the command line 

npm ls -g
/usr/local/lib
├── [email protected]
├── [email protected]

$ node --version
v18.13.0
$ npm --version
9.2.0

I have Prettier and prettier-plugin-toml installed globally.

I try:

prettier --write testconfig.toml

and get:

[error] No parser could be inferred for file "/home/david/Downloads/testconfig.toml".

I also try:

prettier --plugin=prettier-plugin-toml testconfig.toml

and get:

[error] Cannot find package 'prettier-plugin-toml' imported from /home/david/Downloads/noop.js

Am I missing something here, or is there a bug? Thanks.

prettier-plugin-toml cannot ignore lines

In the Prettier Docs it is a general rule to put # prettier-ignore before the token to be ignored.
This does not work with this plugin.

Input

[example]
# prettier-ignore
foo = [
    "bar",
    "baz",
]
hello = [
    "world"
]

Expected Output

[example]
# prettier-ignore
foo = [
    "bar",
    "baz",
]
hello = ["world"]

Actual Output

[example]
# prettier-ignore
foo = ["bar", "baz"]
hello = ["world"]

Workaround

[example]
foo = [  # prettier-ignore, but any comment does the trick
    "bar",
    "baz",
]
hello = ["world"]

TypeScript APIs

  • Lexer
  • Parser
    • Should include the definitions of the CST structure.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.