Did you check existing issues?

• I have read all the tree-sitter docs if it relates to using the parser
• I have searched the existing issues

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

tree-sitter 0.22.1

Describe the bug

I can't figure out how to run tree-sitter generate successfully

Steps To Reproduce/Bad Parse Tree

Tried two methods.

First is the one that works on the upstream. Using node v16.20.2 because node 18+ fails on the yarn installation:

yarn
bash ./scripts/setup-tree-sitter.sh 
yarn tree-sitter generate

With output:

yarn run v1.22.21
$ ./tree-sitter/target/release/tree-sitter generate
/Users/rob/Projects/tree-sitter-yaml/grammar.js:568
  for (const [rule_name, rule] of Object.entries(grammar_json.rules)) {
                                                              ^

TypeError: Cannot read properties of undefined (reading 'rules')
    at global_alias (/Users/rob/Projects/tree-sitter-yaml/grammar.js:568:63)
    at Object.<anonymous> (/Users/rob/Projects/tree-sitter-yaml/grammar.js:470:39)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at [stdin]:420:16
    at Script.runInThisContext (node:vm:129:12)

Node process exited with status 1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Second was to just use the latest tree-sitter (0.22.1, installed with Homebrew):

$ tree-sitter generate
Replacing nan dependency with node-addon-api in package.json
Adding node-gyp-build dependency to package.json
Adding prebuildify devDependency to package.json
Adding an install script to package.json
Adding a prebuildify script to package.json
Adding peerDependencies to package.json
Adding types to package.json
Failed to locate a package.json file that has a "tree-sitter" section, please ensure you have one, and if you don't then consult the docs

Expected Behavior/Parse Tree

I would expect that either the existing yarn tree-sitter generate or a generic tree-sitter generate would work. If not, I would like to see a script or docs for how to build, along the lines of a script in the package.json, a script in the scripts/ directory, or docs in the README.md or CONTRIBUTING.md.

For context, I have a commit that fixes ikatyang#29 with tests here: ikatyang@c1c2506. Happy to submit a PR in this repo with that patch, I'm just not sure how to do that.

Repro

No response

Did you check existing issues?

• I have read all the tree-sitter docs if it relates to using the parser
• I have searched the existing issues

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

[email protected]

Describe the bug

Line that contains key of the mapping cannot be longer than 128 chars. Lexical analysis stops on long line and doesn't continue any further.

More info in ikatyang#59

Steps To Reproduce/Bad Parse Tree

swagger: "2.0"
paths:
  /pet/adding-new-pets/testing-thelong-size-path-name/testing-thepath/testingthepathnameggggggggggggggggggggggggglllllllllllllla: {}
test: 3

Expected Behavior/Parse Tree

The object under paths path will parse as a block mapping key pair and parsing will continue with parsing test: 3.

Repro

ikatyang#59

Did you check existing issues?

• I have read all the tree-sitter docs if it relates to using the parser
• I have searched the existing issues

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

tree-sitter 0.22.5 (cdd46888603e06d6474a96c0024907f68242c45f)

Steps To Reproduce/Bad Parse Tree

As far as I can tell this is valid yaml file:

root: "a\
b\
c"

But it produces a bad tree:

(ERROR
  (block_mapping_pair
    key: (flow_node
      (plain_scalar
        (string_scalar))))
  (ERROR
    (escape_sequence)
    (flow_node
      (plain_scalar
        (string_scalar))))
  (flow_node
    (plain_scalar
      (string_scalar))))

Expected Behavior/Parse Tree

(stream
  (document
    (block_node
      (block_mapping
        (block_mapping_pair
          key: (flow_node
            (plain_scalar
              (string_scalar)))
          value: (flow_node
            (double_quote_scalar
              (escape_sequence)
              (escape_sequence))))))))

Adding additional space after a newline solves the problem somehow:

root: "a\
 b\
 c"

The same'ish problem occurs with arrays and maps: ikatyang#29

Did you check existing issues?

• I have read all the tree-sitter docs if it relates to using the parser
• I have searched the existing issues

Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

No response

Describe the bug

I have been working on this admittedly large 44,000 line file with treesitter-yaml for a few years now with no issues. However with the update to neovim .10 and changes to treesitter I am getting errors with my file. This file is processed by yq and has no reported errors with lsp, yamlfmt or yamllint.

Neovim .10 with up to date parsers

Noevim .10 with nvim-treesitter pinned at 0.9.2

File lexicon.yml located here: https://gitlab.com/ci-dict/dyu-xdxf

And used here: https://dyu-lex.coastsystems.net/dyu

Steps To Reproduce/Bad Parse Tree

1. Open file lexicon.yaml with neovim 10 and up-to-date nvim-treesitter and yaml parser installed.
2. InspectTree will report an errror and highlighting will be wrong
3. Pin nvim-treesitter to 0.9.2 (or maybe 0.9.5)
4. Reinstall yaml query
5. Open file lexicon.yaml (No errro will be reported and highlighting will be correct)

Expected Behavior/Parse Tree

InspectTree should not report errors as the file seems to be syntactically correct.

Repro

https://gitlab.com/ci-dict/dyu-xdxf/-/blob/main/lexicon.yml?ref_type=heads

https://gitlab.com/ci-dict/dyu-xdxf.git