Proposal for supporting semantic highlighting

[timjroberts]

Discussed in #23

^{Originally posted by qobilidop April 20, 2023}
Semantic highlighting is a feature supported by LSP[1], and major editors/IDEs (like VS Code[2]) through LSP. The benefits of semantic highlighting are explained in more detail in [2]. I think it is a very useful feature, and a natural fit for p4analyzer to implement. As a reference, rust-analyzer has implemented this feature already[3].

Reference:

• [1] https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_semanticTokens
• [2] https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide
• [3] Semantic Highlighting rust-lang/rust-analyzer#604
• [4] Grammar of buildfile for VSCode/ide syntax highlighting? build2/build2#109

[vgurevich]

The challenge of building a good semantic highlighter is that it should be architecture-aware, meaning that things like:

• The names of externs (extern constructors) and their parameters
• The names of extern methods and their parameters
• The names of architecture-specific types (typedefs, structs, headers, enums, etc.) and constants
• The names of architecture-specific package (package constructrors) and their parameters

should be properly highlighted. All this information can be gleaned by analyzing the architecture-specific include files, such as tna.p4, psa.p4, etc.

In addition to that, the highlighter should also be target-aware so that it can properly highlight:

• target-specific annotations
• target-specific table attributes (properties)

Unfortunately, there is no standard method to retrieve those. Intel's Tofino compiler allows one to dump all Tofino-specific annotations, but there is no way to dump Tofino-specific table attributes (fortunately, there are not that many).

Semantic highlighter for Emacs supports all the above (@timjroberts -- ask me for the latest version), but only for TNA and T2NA, because all those definitions are hard-coded inside (and I do even update them periodically).

p4analyzer should definitely do better than that.