Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

self keyword not correctly specified / highlighted #3093

Open
BigfootN opened this issue Feb 24, 2025 · 6 comments
Open

self keyword not correctly specified / highlighted #3093

BigfootN opened this issue Feb 24, 2025 · 6 comments

Comments

@BigfootN
Copy link

How are you using the lua-language-server?

NeoVim

Which OS are you using?

Linux

What is the issue affecting?

Other

Expected Behaviour

The self word should be highlighted as a keyword and not a variable.

Actual Behaviour

The self keyword seems to be recognized as a variable and not a keyword. The self keyword is not correctly highlighted. It seems highlighted as a variable and not as a keyword.

Reproduction steps

Use neovim with lua_ls and open a lua file and a theme with the self keyword. The self keyword is not correctly highlighted.

Additional Notes

I first noticed the problem in neovim with the everforest theme. I even opened an issue. More context can be found in the author's detailed answer.

Log File

No response

@BigfootN BigfootN changed the title self keyword not correctly specified self keyword not correctly specified / highlighted Feb 24, 2025
@tomlau10
Copy link
Contributor

It's working correctly in vscode 😕.

And using the Developer: Inspect Editor Tokens and Scopes in vscode shows the following:
Image

  • it's semantic token type is variable.definition:lua
    i.e. it has a definition modifier
  • however my theme doesn't have this rule, so it fallbacks to use textmate scope variable.language.self.lua
    • and my theme seems doesn't have this rule as well, so another fallback => variable.language
    • => finally a rule is matched
  • I can specifically modify the color of self in my user setting json by using:
    • semantic rule: variable.definition:lua
    • or textmate scope: variable.language.self.lua

But all the above are vscode specific, I am not sure the case for neovim 🤔

@antoineco
Copy link

antoineco commented Feb 25, 2025

@tomlau10 no difference with Neovim as far as I can see. self is identified in Neovim as @lsp.type.variable + @lsp.typemod.variable.definition, which matches what you shared from VSCode (type: variable, modifiers: definition).

The rationale here is that self is identified as any other variable, not as a built-in variable, so it can't be highlighted differently from other variables based on LSP Semantic Tokens alone.

The workaround in Neovim is similar to what you shared: clear the highlight for lsp.type.variable when the LSP client is lua_ls, which causes a fallback to the next method: Tree-sitter if enabled, otherwise Vim's legacy syntax.

Disclaimer: I'm not very familiar with LSP Semantic Tokens so I am not even sure whether language servers have a notion of "built-in" tokens. Tree-sitter does, apparently Textmate too.
The only thing that seems to come close is the defaultLibrary modifier, but I'm not sure whether that's appropriate for self because it matches on a lot of other things.

@tomlau10
Copy link
Contributor

tomlau10 commented Feb 25, 2025

Disclaimer: I am not maintainer of LuaLS, nor have rich knowledge related to semantic tokens 🙈


I am not even sure whether language servers have a notion of "built-in" tokens. Tree-sitter does, apparently Textmate too.

I believe the token types are given by vscode's LSP spec 🤔
https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide#standard-token-types-and-modifiers
=> There is no builtin type or builtin modifier

You can see that LuaLS also defined TokenType enum in that way:

m.TokenTypes = {
["namespace"] = 00,
["type"] = 01,
["class"] = 02,
["enum"] = 03,
["interface"] = 04,
["struct"] = 05,
["typeParameter"] = 06,
["parameter"] = 07,
["variable"] = 08,
["property"] = 09,
["enumMember"] = 10,
["event"] = 11,
["function"] = 12,
["method"] = 13,
["macro"] = 14,
["keyword"] = 15,
["modifier"] = 16,
["comment"] = 17,
["string"] = 18,
["number"] = 19,
["regexp"] = 20,
["operator"] = 21,
["decorator"] = 22,
}

The rationale here is that self is identified as any other variable, not as a built-in variable

From the above vscode's doc, you can see that semantic token types are always used together with modifier type.
So I don't think you can just do theming based on token type while ignoring its modifier? 😳
For example, you can have:

  • variable.global
  • variable.readonly
  • variable.static
  • ...

=> all the above are variables but with different attributes (modifier)
And now LuaLS is giving self as variable.definition => which can be interpreted as a built-in variable (I guess)


Can your theme support coloring based on type + modifier 🤔 ?
More specifically a rule for @lsp.type.variable + @lsp.typemod.variable.definition ?
(I am not familiar with neovim)

The workaround in Neovim is similar to what you shared: clear the highlight for lsp.type.variable when the LSP client is lua_ls, which causes a fallback to the next method: Tree-sitter if enabled, otherwise Vim's legacy syntax.

If you prefer your editor's default highlight, here is another workaround:

@tomlau10
Copy link
Contributor

I know that @lewis6991 is a pro user and lsp client developer in neovim,
maybe he could give you some insight on this problem 👀

@antoineco
Copy link

That's what I expected, thanks @tomlau10.

Neovim applies higher priorities to types with modifiers, so you can actually be granular, but variable.definition is still quite broad despite having a modifier (a variable definition doesn't necessarily mean a built-in).

Your workaround makes sense. In fact it is similar to what I suggested @BigfootN (OP) to do.

So to summarize, if semantic tokens aren't expressive enough for the language with regards to syntax highlighting, either:

  • Configure Lua LS to disable semantic tokens for that type (or entirely)
  • Clear the corresponding highlight if it is set in your editor/colorscheme to fall back to the next syntax method

Issue can be closed, I guess?

@BigfootN
Copy link
Author

BigfootN commented Feb 25, 2025

Hello,

Sorry for responding so late, and thank you all for all your detailed explanation.

Indeed, the workaround @antoineco suggested to me seems to "solve" the problem.

So if I understand well, the theme should support this (from @tomlau10 ):

And now LuaLS is giving self as variable.definition => which can be interpreted as a built-in variable (I guess)

Should I close this issue ?

Thank you again for all your precious help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants