Surprising behaviour of default 0.10 color scheme groups: `Added`, `Changed`, `Delimiter`, `Operator`, `Removed` and `@variable` about neovim HOT 12 OPEN

Thanks for the discussion. I can understand the reason for the new Added, Changed, and Removed now.

no group name is marked with * in that list

Yes, I see that there is no asterisk, but as was pointed out above, this was silently removed at some point due to a rendering issue. I assumed that the classification of groups had remained the same, based on the visual grouping in :h group-name. I stand corrected.

Adding NeoVim-only highlight groups was already necessary, that is true, but the listings at :h diagnostic-highlights and :h lsp-highlight makes these trivial to support. Actually, the LSP ones link to standard group-names by default. My main gripe is with the silent promotion of tree-sitter highlight groups to a "default" level, such that colorscheme maintainers now have to add definitions for yet another list of symbols (which I just found at :h treesitter-highlight-groups), or have the most basic semantics of their colorschemes break.

But, I'm getting the message that this is unavoidable, so I will distill the essence of my complaints in that case:

• Treesitter is called "experimental" in :h treesitter, which makes it surprising to me that treesitter highlights are now essential for NeoVim 0.10 colorschemes
• Turning hi clear, which is common practice, into a landmine felt a little brutal and was not really signposted IMO; I could like a note at :h :hi-clear that mentions where to find the default definitions that will be used (which requires that these be defined in a predictable place)
• The list of required highlight groups for a minimal colorscheme has now expanded at least 2x compared to Vim8, and the relevant symbols are not trivially discoverable (they are split among at least the above 3 locations in the helpfiles)

from neovim.

You identified a number of documentation improvements that I plan to tackle, so we can keep this open until I find time.

from neovim.

disclaimer: I'm not the author of the new default colorscheme.

Why is Added not linked to DiffAdd

You can use diff mode with diff files.

Most surprisingly, why is @variable not linked to e.g. Identifier?

Then pretty much everything will be highlighted blue in the default colorscheme. Not sure if that's what people like.

this change should be reflected in :h group-name, with those names being singled out instead of nested under Special and Statement by default

I concur. In fact, :h group-name explicitly mentions that the grouping of highlight groups implies the default link:

The names marked with * are the preferred groups; the others are minor groups.
For the preferred groups, the "syntax.vim" file contains default highlighting.
The minor groups are linked to the preferred groups, so they get the same
highlighting.  You can override these defaults by using ":highlight" commands
after sourcing the "syntax.vim" file.

(* thing is deleted in #24087 for an unrelated reason)

from neovim.

This is more of a documentation issue (taken from Vim but not corrected): We don't make the same distinction of "preferred" and "minor" groups -- the "treesitter" groups are just as official as the legacy groups. (And as you can see, no group name is marked with * in that list.)

Why is Added not linked to DiffAdd
You can use diff mode with diff files.

This.

If it is really necessary to have Delimiter and Operator as distinct highlights by default, this change should be reflected in :h group-name, with those names being singled out instead of nested under Special and Statement by default

I don't understand this.

if support for Vim8 themes is still intended.

Not without change; themes targeting Vim8 (only) were broken anyway since they lacked a number of Neovim-only highlight groups (in particular for treesitter, LSP, and diagnostics), and the legacy "preferred" groups were just too limited to cover these with links in a reasonable way.

TL;DR: It is intended for colorschemes to support both Vim8 and Neovim; it is not intended that Vim8-only colorschemes "just work" in Neovim.

from neovim.

I assumed that the classification of groups had remained the same, based on the visual grouping in :h group-name. I stand corrected.

No worries; I missed that as well. I'll edit the docs to remove the confusion.

Treesitter is called "experimental" in :h treesitter, which makes it surprising to me that treesitter highlights are now essential for NeoVim 0.10 colorschemes

As a whole, yes; but parts of it are becoming stabilized so we incrementally "graduate" them -- this includes highlighting (at least for "core" languages), as part of which the treesitter highlight groups were upstreamed and "made official". :h news should have called that out.

Turning hi clear, which is common practice, into a landmine felt a little brutal and was not really signposted IMO; I could like a note at :h :hi-clear that mentions where to find the default definitions that will be used (which requires that these be defined in a predictable place)

That is unfair. hi clear works as before; it just (now) resets to the new default colorscheme -- which, again, was called out in :h news. This also links to the compatibility step: :color vim after :hi clear and before setting the new groups.

The list of required highlight groups for a minimal colorscheme has now expanded at least 2x compared to Vim8, and the relevant symbols are not trivially discoverable (they are split among at least the above 3 locations in the helpfiles)

Yes; you also get more than 2x the features -- a fair trade, I'd say. I agree that the increasing split (note that the "official" highlight groups were split already before!) is not great; I'll see how this can be improved by adding a list of links in an appropriate place.

TL;DR: I absolutely agree that documentation can be improved (and should have been before the 0.10 release; alas, the timing made that infeasible), but I can't agree that the change itself is offensive.

from neovim.

On reflection, I agree that my previous comment comes across a bit too harsh in places. I'm glad that writing this up has led me to discover where the relevant listings are in the helpfiles, and maybe other readers will find that information useful for v0.10. Thanks for taking the feedback on board, I do apreciate that maintaining documentation is often a thankless task.

from neovim.

And I am grateful to you for pointing out the "preferred groups" bit, which I genuinely missed.

from neovim.

Going through and adding NeoVim definitions to my colorscheme, I came across a minor related point: Some treesitter groups (and possibly LSP ones too) cannot be inspected with :hi, only by doing :Inspect on the relevant line in the helpfile. With the default colorscheme, one example is :hi @variable.parameter, which upon execution returns nothing. :Inspect tels me that it is linked to Identifier. It would be nice to have these working with :hi as well, if possible.

Update: It seems that this affects all syntax groups that "exist" but don't show up in the paged output of :hi.

from neovim.

Update: It seems that this affects all syntax groups that "exist" but don't show up in the paged output of :hi.

The issue is that these don't exist. Instead, the highlighter uses the implicit fallback to @variable (which does exist). This is exactly why all these groups are now highlighted in help; :Inspect is the correct tool for this, not :hi.

from neovim.

Understood. In fact, that works well: nvim -u NORC, followed by packadd <my_colorscheme> and then :hi seems like a good way to show a relatively concise list of things that a colorscheme maintainer should implement in that case. Perhaps I took the wrong approach in trawling through helpfiles in the first place. I'm not sure how useful it is to keep this open in that case, but I'll leave it up to you.

from neovim.

I got pretty confused by this too, and I'm still trying to sort out all of the changes that need to happen to unbreak my color theme. I did find one change though that seems to go against the documentation mentioned here as well and figured maybe I should mention it here?

This doesn't seem like it should be linked to Constant, a macro should be linked against PreProc or Macro (probably Macro would make more sense here?) shouldn't it?

from neovim.

It's linked to @constant.macro; feel free to link that to either of these. (The builtin color scheme is intentionally minimal so it can't distinguish between these two. Other color schemes should not assume any links and just define the groups as they deem fit.)

from neovim.