dvdsk / prosesitter.nvim Goto Github PK

Offer to auto install vale (probably using a bash script, see how lspinstall does things)

(check on lintreq.lua:102 fails)

input: a medium size latex file (my report on parallel MST construction)

docstrings can not be recognised using the query API (I cant think of a good way and neither can others it seems nvim-treesitter/nvim-treesitter#923). Therefore we need to handle this in a preprocessing function.

• Add preprosessing function in /lua/prosesitter/preprocessing.lua to recognize docstrings.
• Device a method to allow users to enable only parsing docstrings.

Deleting lines then undoing that gives extmark could not be placed error

minimal example:

\documentclass{article}
\begin{document}

Vales \href{https://github.com/errata-ai/styles}{style libary}. 

\end{document}

In on_event.attach and soon in on_event.on_lines we need to assure we have an up to date treesitter tree to work on. To get it I force to parse the tree. Parsing sounds expensive however.

TODO

• how expensive is force parsing?
• can we delay till parsing is done as it is done anyway for highlighting

Example:

/* sm spll errs */

fn main() {
    println!("sm spll errrs")
}

This leads to the following highlighting if one types the part within the block comment:

However after saving and loading the file the errors are highlighted correctly. Editing the block comment also works fine.

test file: prosesittes.lua

error is coming from languagetool: Error: Could not decode query. Query length: 613 Request method: POST

comes from this_line

(non minimal) example:

#include "util.hpp"
#include <limits>
#include <random>
#include <stdlib.h>
#include <algorithm>

using std::vector;

vector<uint32_t> random_arry(int n, int seed) {

	std::mt19937 rng(seed);

	auto max = std::numeric_limits<uint32_t>::max();
	std::uniform_int_distribution<uint32_t> dist(0, max);
	auto generator = [&dist, &rng](){ return dist(rng); };

	vector<uint32_t> data(n);
	std::generate(data.begin(), data.end(), generator);
		
	// return value optimization will prevent copy
	return data;
}

// checks if data in array is sorted in ascending order
bool is_sorted(vector<uint32_t> data) {
	auto prev = 0;
	for(const auto& n : data) {
		
	}
}

is sorted is highlighted and PsDisable gives a stack traceback:

E5108: Error executing lua ...start/prosesitter/lua/prosesitter/linter/check/check.lua:45: attempt to index local 'self' (a nil value)
stack traceback:
        ...start/prosesitter/lua/prosesitter/linter/check/check.lua:45: in function 'cancelled_schedualled'
        ...start/prosesitter/lua/prosesitter/linter/check/check.lua:63: in function 'disable'
        ...er/start/prosesitter/lua/prosesitter/linter/on_event.lua:140: in function 'disable'
        ...m/site/pack/packer/start/prosesitter/lua/prosesitter.lua:50: in function 'disable'
        [string ":lua"]:1: in main chunk

on opening an empty file

Instead of strings, comments allow:

• strings
• comments
• doc comments

and any combination of these

reproduce: open popup using , then click in any window with the mouse.

solution: also map mouse clicks to close window, see: <LeftMouse> <RightMouse>

Would not lint text that matches the name of one of the symbols in the code.

The check itself can be done fast if a Lua table exists of all symbols known to tree-sitter then we only need to check for each word if it is in the table.

The problem is getting such a table as it needs to be up to date each time we start checking a new piece of text. For now the only way I see we could do this is adding a tree-sitter query that extracts symbols and updates the table. Taking care to remove items that no longer exist

quite a lot of functionality was added, should refactor to keep maintainable.
list of ideas:

• module on_event should be renamed, maybe linter?

• handle vale and language tool checking the same code better
  example problem: Sentence start without capital detected by language-tool
• refactor the code, related to: #27
• language-tool is spinning up a 10 thread thread-pool... probably somewhat overkill
• language-tool port should be configurable

to reproduce try:

// hello hello 

Happens again if more letters are added.

issues.lua:63

Probably caused by us not cleaning up ext-marks properly. Problem is that Nav (PsNext) gets an outdated mark without the correct issuemeta attached

test file: on_event.lua from this repo.

A debug flag would allow for more rigorous defensive programming practices without having to worry about performance. Suggested asserts:

• no duplicates in lintreq
• linter problems are sorted by Span
• overlapping regions in lintreq

It would be nice to lint a block of comments with its context. For example changing line 2 should take into account line 1 and 3 here:

// This is a single
// sentence and should
// be linted as one.

This will be hard to implement as it requires:

• getting the context, it is unclear if a block comments is returned as a single node in vim highlighting
• removing leading comment signs
• removing line break

If I get requests for this, I have time and/or really need this myself I will implement it.

do this in separate branch, should features:

• list of features
• install instructions
• config options
• gif/ascii cinema of plugin in action

check what other vim plugins do

Since math is not prose

related question on Treesitter discussion forum tree-sitter/tree-sitter#1445

will become a more serious problem with the introduction of #5

given the following latex for a misspelling of longest with the letters ng italic

Lo\textit{ng}este

Vale will want to highlight the entire word for a spelling mistake however will only underline Lo\texti as that is the length of the error in the text vale got to process

Test file: https://github.com/dvdsk/mock-fs/blob/main/meta-server/src/main.rs

could be useful to forbid checking a certain comment or for example a tikzpicture in a latex document

This is my prosesetter.lua file

require("telescope").load_extension("prosesitter") -- Optionally, depends on telescope.nvim
require("prosesitter"):setup({
	vale_bin = vim.fn.stdpath("data") .. "/user/bin/vale",
	vale_cfg = vim.fn.stdpath("data") .. "$HOME/.config/vale/.vale.ini",
	--optional extra queries overrides existing queries
	queries = {
		-- see the piece on adding queries on how to use this
		-- (not needed if using an out of the box supported language
		py = {
			strings = "[(string) ] @capture",
			comments = "[(comment)+ ] @capture",
		},
	},
	-- highlight groups to use for lint errors, warnings and suggestions
	severity_to_hl = { error = "SpellBad", warning = "SpellRare", suggestion = "SpellCap" },
	-- weather to lint strings, comments or both for a language
	lint_targets = { py = "both", tex = "strings", sh = "comments" },
	disabled_ext = { "tex" }, -- do not ever lint tex files
	auto_enable = false, -- do not start linting files on open (default = true)
	default_cmds = false, -- do not add commands (default = true)
})

I am getting the following error

Language tool not installed, install language tool? y/n: yError detected while processing /home/aaron/.dotfiles/nvim/.config/nvim/init.lua:
E5113: Error while calling lua chunk: ...ck/packer/start/impatient.nvim/lua/impatient/profile.lua:146: ...e/pack/packer/start/prosesitter/lua/prosesitter/util.
lua:16: Vim:E492: Not an editor command: IndentBlanklineRefresh

The language tool is already installed but it will still ask to be installed. Neverthless....

Error executing vim.schedule lua callback: ...ker/start/prosesitter/lua/prosesitter/linter/lintreq.lua:107:attempt to index local 'meta' (a nil value)

Probably present since adding support for multiple nodes on same line

Urls and paths being marked as spell errors is currently an issue for latex and most programming languages

Text to be checked/linted is added to a lintreq. To determine whether text should be linted I check if it corresponds to an allowed highlight group using tree-sitter queries, a allow list based system:

For any tree-sitter subtree (parser:for_each_tree)
     iter over highlights (hl_query:iter_captures)
         add to lintreq if allowed hl_group

Some languages, such as latex, need text linted that is not highlighted. For this we need a deny list style operation.

change

Configuration must be language dependent
Instead of always running in allow list mode the event handler on_event should change to deny list mode depending on config

language dependent config

options:

• switch based on parser lang
• switch based on buffer lang (an autocommand?)
• switch based on file extension (see 25ec193)

deny list style on_event

split lintreq builder into allow builder and deny builder. Both still build into a lintreq

Typing one character triggers 4 calls to on_lines, this is wasteful. Tested using deny list based linting.

languagetool sometimes goes beserk in its output giving MANY spelling replacement options this result in an error like:

Error executing vim.schedule lua callback: Vim:E474: Trailing characters: },{"value":"SSAP"},{"value":"SSB"},{"value":"SSF"},{"value"
:"SSG"},{"value":"SSH"},{"value":"SSI"},{"value":"SSK"},{"value":"SSPI"},{"value":"SSS"},{"value":"SSV"},{"value":"ST"},{"value":"SU"
},{"value":"SW"},{"value":"SY"},{"value":"SZ"},{"value":"Sb"},{"value":"Sc"},{"value":"Se"},{"value":"Sep"},{"value":"Si"},{"value":"
Sm"},{"value":"Sn"},{"value":"Sq"},{"value":"Sr"},{"value":"TAP"},{"value":"TBP"},{"value":"TDP"},{"value":"TEP"},{"value":"TGP"},{"v
alue":"THP"},{"value":"TIP"},{"value":"TKP"},{"value":"TLP"},{"value":"TMP"},{"value":"TNP"},{"value":"TOP"},{"value":"TP"},{"value":
"TPP"},{"value":"TRP"},{"value":"TS"},{"value":"TSB"},{"value":"TSC"},{"value":"TSD"},{"value":"TSE"},{"value":"TSG"},{"value":"TSH"}
,{"value":"TSI"},{"value":"TSK"},{"value":"TSL"},{"value":"TSM"},{"value":"TSN"},{"value":"TSO"},{"value":"TSR"},{"value":"TSV"},{"va
lue":"TSVP"},{"value":"TSX"},{"value":"TTP"},{"value":"TUP"},{"value":"TVP"},{"value":"UAP"},{"value":"UFP"},{"value":"UIP"},{"value"
:"ULSI"

The values here are suggested replacements. the actual error is trailing characters possibly occured during json decode?

Should we move the call of the global _G.ProseSitter into the api? enabling the user to call require("prosesitter").enable() instead of _G.ProseSitter.enable()?

Pros:

• more inline with rest of ecosystem
• can explicitly check if setup was ran and give appropriate error

Conns:

• perf hit, probably not even measurable

TODO hook into undo event to schedule a check

Originally posted by @dskleingeld in #11 (comment)

to replicate print something in the mark results window then really slowly type and see how a pop window triggers multiple print messages

Current approach is to use a default query as fallback to language specific queries. However tree-sitter nodes are not called the same across all languages.

Solution: implement queries for all languages

Either load from path or string. Should be a function so it can be scripted from a config (switching config on a key press for example)

alert error:
no linked issue id,linter,buf: 3,vale,1
source marks.lua:91

[INFO  linter/marks/marks.lua:103: *** mark results
[INFO  linter/marks/marks.lua:47: ---->removing old marks
[INFO  linter/marks/marks.lua:40: {}
[INFO  linter/marks/marks.lua:40: {}
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col langtool 1 -0 196 201
[DEBUG linter/marks/marks.lua:80: added mark, id,linter,buf: 1 langtool 1
[DEBUG prosesitter/linter/issues.lua:96: adding issue list for id,linter,buf: 1 langtool 1
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col langtool 1 -0 282 287
[DEBUG linter/marks/marks.lua:80: added mark, id,linter,buf: 2 langtool 1
[DEBUG prosesitter/linter/issues.lua:96: adding issue list for id,linter,buf: 2 langtool 1
[INFO  linter/marks/marks.lua:103: *** mark results
[INFO  linter/marks/marks.lua:47: ---->removing old marks
[INFO  linter/marks/marks.lua:40: {}
[INFO  linter/marks/marks.lua:40: { { 1, 0, 196, {
      end_col = 201,
      end_row = 0,
      hl_group = "SpellBad",
      priority = 4096
    } }, { 2, 0, 282, {
      end_col = 287,
      end_row = 0,
      hl_group = "SpellBad",
      priority = 4096
    } } }
[DEBUG linter/marks/marks.lua:13: possibly clearing mark id,linter,buf 1 vale 1
[DEBUG prosesitter/linter/issues.lua:80: removing issue list for id,linter,buf: 1 vale 1
[DEBUG linter/marks/marks.lua:13: possibly clearing mark id,linter,buf 2 vale 1
[DEBUG prosesitter/linter/issues.lua:80: removing issue list for id,linter,buf: 2 vale 1
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col vale 1 1 48 51
[DEBUG linter/marks/marks.lua:80: added mark, id,linter,buf: 3 vale 1
[DEBUG prosesitter/linter/issues.lua:96: adding issue list for id,linter,buf: 3 vale 1
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col vale 1 -0 170 180
[DEBUG linter/marks/marks.lua:80: added mark, id,linter,buf: 4 vale 1
[DEBUG prosesitter/linter/issues.lua:96: adding issue list for id,linter,buf: 4 vale 1
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col vale 1 -0 170 172
[DEBUG linter/marks/marks.lua:80: added mark, id,linter,buf: 5 vale 1
[DEBUG prosesitter/linter/issues.lua:96: adding issue list for id,linter,buf: 5 vale 1
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col vale 1 -0 196 201
[INFO  linter/marks/marks.lua:88: { 1, 0, 196, {
    end_col = 201,
    end_row = 0,
    hl_group = "SpellBad",
    priority = 4096
  } }
[DEBUG prosesitter/linter/issues.lua:90: retrieving issue list for id,linter,buf: 1 langtool 1
[DEBUG prosesitter/linter/issues.lua:96: adding issue list for id,linter,buf: 1 vale 1
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col vale 1 -0 223 230
[DEBUG linter/marks/marks.lua:80: added mark, id,linter,buf: 6 vale 1
[DEBUG prosesitter/linter/issues.lua:96: adding issue list for id,linter,buf: 6 vale 1
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col vale 1 -0 282 287
[INFO  linter/marks/marks.lua:88: { 2, 0, 282, {
    end_col = 287,
    end_row = 0,
    hl_group = "SpellBad",
    priority = 4096
  } }
[DEBUG prosesitter/linter/issues.lua:90: retrieving issue list for id,linter,buf: 2 langtool 1
[DEBUG prosesitter/linter/issues.lua:96: adding issue list for id,linter,buf: 2 vale 1
[INFO  linter/marks/marks.lua:74: !!!! linter,buf,row,start_col,end_col vale 1 1 48 51
[INFO  linter/marks/marks.lua:88: { 3, 1, 48, {
    end_col = 51,
    end_row = 1,
    hl_group = "SpellCap",
    priority = 4096
  } }
[DEBUG prosesitter/linter/issues.lua:90: retrieving issue list for id,linter,buf: 3 langtool 1

example demo.tex change capitalization of the third sentence

Found during writing of remove_placeholders function

This is probably bad :), though not trivial to fix since we do not want to delete a placeholder that has been updated. Solution only remove if mentioned in a lintreq.

example:

\documentclass{article}
\begin{document}

this [I] to {capture} this
hello (test) \{\{hi\}\}

\end{document}

text passed: this I to capture this hello test hi

does not go to the previous mark reliably

example, the second word is clearly a spell error and was highlighted before

might also take the form of a separate plugin.

or a port of https://github.com/Ron89/thesaurus_query.vim to lua, removing the python dependency or replacing it with a rust binary?

tracking-branch: langtool

• hard coded path run check in parallel to vale api
• should combine output of vale and language-tool after checking
• start language server (if not running) on setup
• offer to install languagetool if not found, check how grammarous does auto install
• make lang tool integration optional

Cause probably related to #11, solutions:

• allow error when setting highlights (use pcall); disadvantage when editing a problem the word will not be underlined.
• adjust the end column so the edit can still be placed; however hides that we are using an now outdated result
• adjust the color to reflect out dated result and apply; however might be distracting to see color changes
• leave old edit in place; disadvantage complexer to implement

prosesitter.nvim/lua/prosesitter/on_event/marks/marks.lua:55: end_col value outside range

look at https://github.com/lewis6991/spellsitter.nvim/blob/ca3224a9184507345aa37baa08d9f542e01dcdc9/queries/python/spell.scm for a better docstring query