Language service reimagined

[tomrav]

A New LSP on the Horizon (WIP)

This is currently blocked until our efforts on the selector and value parsers is complete

A language service designed to run in both Monaco and VSCode, providing language capabilities for Stylable.

See capabilities description below.

Capability priorities and high level tasks

1. publishDiagnostics - business logic implemented in @stylable/core
   • Stylable functionality already exists
   • validate native css
2. completion
   • deps: improved parsing
   • ...
3. definition, implementation, references, typeDefinition, declaration
   • deps: improved parsing,
4. hover
   • selectors
     • deps: none
     • specificity calculator
     • dom structure snapper
     • transformed value
   • properties
     • deps: aggregate CSS metadata
     • description, top level syntax, link to MDN
     • css custom property
       • potential value
       • type
       • origin
       • global
   • value
     • deps: improved parsing (value parser), aggregate CSS metadata
     • info on part of value (syntax, link to MDN)
     • native functions (syntax, link to MDN)
     • CSS custom property / Stylable variables
       • potential value
       • type
       • origin
       • global
     • mixin
       • transformed snippet (with overrides)
     • file paths / <url()>
       • resolved absolute file paths
       • server relative path can be tricky
   • stylable directives
     • aggregate information for all Stylable custom directives
       • description
       • syntax
       • link to stylable.io
5. semantic tokens
   • deps: improved parsing (value parser)
   • research CSS AST token type and modifier mapping
6. formatting
7. signatureHelp
   • deps: improved parsing (value parser), context for location, JS mixin and formatter analysis
8. documentColor, colorPresentation
   • deps: improved parsing (value parser), context for location
9. rename
10. documentHighlight
11. documentSymbol
12. codeAction
13. codeLens
14. codeLens refresh
15. selectionRange
16. linkedEditingRange
17. foldingRange
18. documentLink

High level tasks

• improved parsing - custom parsers needed for . a lot of capabilities are used when the stylesheet input is broken, all behaviors must be safe and not hinder user experience
  • selector parser
    • implement + use
  • value parser
    • syntax parser
    • properties and at-rules
      • syntax definition list
      • tooling (validation, classification, etc.)
  • resilient input handling in edit mode
• context for location - resolve contextual information from location data
• smart completions - support a fuzzier approach and sort according to some logic
• Stylable version selection - let the user choose between the latest version of Stylable (included with the extension) or the version they're using in their project
• JS mixin and formatter analysis - determine argument typings for signature hinting
• symbol locator - given certain search criteria return a list of matching, available symbols
• aggregate CSS metadata
  • dom elements information - might be good for generating native css dom lib: Native DOM Typings #1985
    • inheritance, pseudo-elements, pseudo-classes
    • description
    • mdn link
  • values - at-rules, properties, functions
    • description
    • mdn link

completion

Below is a mapping of all completions in which Stylable plays a part (not native CSS).
The mapping is broken down into AST parts (ish), at-rules, rules and declarations.

For each completion, its location and completion type are detailed.

location - is a list of all allowed places for this completion to appear, each completion should not appear outside of its allowed locations

completion - two options, either a general description of things that need to be computed (like, available symbols) or a template string representing the completion format, with the cursor tab-jumping to the next "$" in a descending order ($0 is implicitly the end of the string unless explicitly defined otherwise)

do we suggest directives that are being deprecated? (e.g. @namespace)

• at-rules
  • @st-import
    • directive itself
      • location: top-level
      • completion: @st-import $1 from "$2";
    • [named...] inner parts - available symbols, keyframes()
      • location: @st-import inner directive
      • completion: available symbols | keyframes($1)
    • keyframes(xxx) inner parts - available keyframe symbols
      • location: @st-import named part
      • completion: keyframe symbols
    • from - relevant path suggestions
      • location: @st-import directive
      • completion: path or package request
  • @st-namespace (or @namespace)
    • directive itself
      • location: top-level, once per stylesheet
      • completion: @st-namespace "$1";
  • @st-global-custom-properties
    • directive itself
      • location: top-level
      • completion: @st-global-custom-properties --$1;
    • suggested variables in scope
      • location: @st-global-custom-properties directive
      • completion: known css variable names (not symbols, not imports!)
  • @keyframes
    • st-global(xxx) - keyframe name wrapper
      • location: @keyframes definition
      • completion: st-global($1)
  • @st-scope
    • directive itself
      • location: top-level
      • completion: @st-scope $2 {\n $1\n}
    • scoping selector - selector used to scope inner scope content
      • location: @st-scope directive
      • completion: selector
  • @st-custom-selector (or `@custom-selector)
    • directive itself + :--
      • location: top-level
      • completion: @st-custom-selector :--$2 $1;
    • selector value
      • location: @st-custom-selector directive
      • completion: selector
  • @property
    • st-global(--xxx) - css var name wrapper
      • location: as part of a @property definition
      • completion: st-global(--$1)
  • @media (explore other native at-rules)
    • value(xxx) - in the params
      • location: as part of a @media definition
      • completion: value($1)
  • @st-vars (does not exist yet)
    • directive itself
      • location: top-level
      • completion: @st-vars $2 {\n $1\n}
    • custom-type directives - st-map, st-array and custom types (e.g. stBorder when imported), including nesting
      • location: declaration values inside @st-vars
      • completion: st-map($1) | st-array($1) | st-border($1)
    • value suggestions in single var, and st-map/st-array values
      • location: inside custom-values directives
      • completion: any declaration value | string
• rules
  • :import
    • directive itself
      • location: top-level
      • completion: :import {\n $1\n}
    • inner directive - st-from, st-default, st-named
      • location: inside :import directive
      • completion: -st-from: "$1"; | -st-default: "$1"; | -st-named: "$1";
    • -st-named inner parts - available named symbols, keyframes()
      • location: inside -st-named directive
      • completion: available symbols from imported scope
    • -st-from - relevant path suggestions
      • location: inside -st-from directive
      • completion: path or package request
  • :vars
    • see @st-vars
      • completion: :vars {\n $1\n}
  • selectors
    • :st-global() (or :global())
      • directive itself
        • location: inside any selector
        • completion: :st-global($1)
      • nested selector content
        • location: inside st-global()
        • completion: selector
    • classes - .part
      • location: inside any selector
      • completion: class symbols in scope
    • elements - Comp
      • location: inside any selector
      • completion: element symbols in scope
    • pseudo-classes
      • boolean state - :userSelected
        • location: inside any selector, following a symbol that has an appropriate state defined
        • completion: state symbols from the scope of the last selector subject
      • state with param - :userSelected(xxx)
        • state name with parenthesis
          • location: inside any selector, following a symbol that has an appropriate (enum) state defined
          • completion: valid enum state values
        • value(xxx) as param (considering deprecation)
          • location: inside a state with a param
          • completion: value($1)
    • pseudo-elements
      • ::myPart - according to selector context
        • location: inside any selector, following a symbol with an inner part defined
        • completion: part symbols from the scope of the last selector subject
    • custom-selector - available :--xxx parts
      • location: inside any selector, following a symbol with a custom selector defined
      • completion: custom selector symbols in scope
    • .root - inherently exists in every stylesheet (probably handled by .classes completion)
      • location: inside any selector
      • completion: .root {$1}
• declarations
  • directives
    • -st-extends
      • directive itself
        • location: declaration inside the a simple rule, once per rule
        • completion: -st-extends: $1;
      • value of a single class/element (multiple in the future)
        • location: value of the -st-extends declaration
        • completion: classes or element symbols in scope
    • -st-states
      • directive itself
        • location: declaration inside a class rule, once per rule
        • completion: -st-states: $1;
      • state type - myState(xxx)
        • location: value of the -st-states declaration, inside a state definition with parenthesis
        • completion: string | number | tag | enum()
      • state param validators - myState(string(minLength(5)))
        • string
          • location: inside a state string type definition with parenthesis
          • completion: minLength | maxLength | contains | regex
        • number
          • location: inside a state number type definition with parenthesis
          • completion: min | max | multipleOf
    • -st-mixin
      • directive itself
        • location: declaration inside any rule, once per rule
        • completion: -st-mixin: $1;
      • mixin name
        • location: value of the -st-mixin declaration
        • completion: class or element symbols available in scope
      • CSS
        • var override keys - suggesting vars used in mixin scope
          • location: inside a CSS mixin with parenthesis, left side of the key/value
          • completion: Stylable variable names used in the mixed in scope
        • var override values - suggest values
          • location: inside a CSS mixin with parenthesis, right side of the key/value
          • completion: any declaration value | string
      • JS
        • value(xxx) as nested arguments
          • see value(xxx, yyy)
    • -st-global
      • directive itself
        • location: declaration inside any rule
        • completion: -st-global: $1;
      • value is a selector;
        • location: declaration value of the -st-global directive
        • completion: selector
  • values
    • value(xxx, yyy)
      • directive itself
        • location: any declaration value
        • completion: value($1)
      • where xxx is the name of a Stylable variable in scope
        • location: first argument inside a value() directive in a declaration value
        • completion: Stylable variables available in scope
      • where yyy is a list of comma-separated member accessors (keys) on the var based on its type (optional)
        • location: rest of arguments inside a value() directive of the appropriate type in a declaration value
        • completion: Stylable variable key names / indexes
    • st-global(xxx) (future)
      • keyframes
        • location: wrapping keyframe name in animation or animation-name declaration values
        • completion: st-global($1)
      • CSS vars
        • location: wrapping css variable usage (inside var())
        • completion: st-global($1)
    • formatters
      • value(xxx) as nested arguments
        • see value(xxx, yyy)

hover

• selectors
  • including: states, pseudo-elements, custom-selectors, st-scope
  • specificity?
• value(xxx)
  • resolved value
• any symbol (especially imported ones)
  • type and origin
• path requests
  • resolved path
• Stylable directives descriptions
  • at-rules
    • @st-import
    • @st-namespace
    • @st-custom-global-properties
    • @st-scope
    • @st-custom-selector
    • @st-vars
  • rules
    • :import
    • :global - include in selectors?
    • .root - special case?
  • declarations
    • -st-extends
    • -st-states
    • -st-mixin
    • -st-global
    • -st-from
    • -st-default
    • -st-named
  • values
    • st-global()
    • value() - see value(xxx) above

signatureHelp

• Stylable directives
  • st-global() - contextually aware (var/keyframe)
  • value(x, y, z) - according to var type
  • custom-value definition - st-map, st-array and custom ones
  • keyframes() - in named import statements
• states with param - param type hinting
• formatter - argument hinting
• mixin - argument hinting
  • CSS
  • JS

Native CSS

• maintain all native input lists - elements, properties, keywords, etc.
• additive / conflicting APIs between native CSS and Stylable files - some APIs can be nicely composed (e.g. completions), but there's a big difference in how the LSP logic handles CSS vs Stylable, even for those APIs. If we decide to handle native CSS ourselves, we will need to map out all such differences

Infra / Host / Configuration

• browser compatible - needs to be able to run in Monaco

monaco vs. vscode

Some differences exists between the two LSP possible consumers, needs further research.

• completion cursor location - marking completion locations in monaco and vscode differ (using "$" in vscode, unknown in monaco)
• syntax highlighting - vscode uses textmate syntax, monaco uses monarch (semantic tokens may work for both cases, needs validating)
• position vs. offset - vscode and monaco use different formats for indicating locations and ranges, need a set of utilities to easily traverse the two when needed

Testing

• e2e tests - using vscode-test to test general feature flows
• unit/ix - for more granular feature testing, testing can take place on the "protocol" or directly on the APIs themselves

---

Research References

• typescript-lanaguage-server
• vscode-css-languageserver
• vscode-css-languageservice
• list of many LSP implementation

[tomrav]

Capabilities Description

Demo for most language features

publishDiagnostics

Diagnostics are a way to indicate issues with the code.

For Stylable: info/warning/error diagnostics emitted by Stylable for any *.st.css file

completion

Code completions provide context sensitive suggestions to the user.

For Stylable: completions for Stylable symbols and contexts

completion resolve

Support resolve providers that compute additional information for completion proposal the user selects. This information is displayed along-side the selected item.

For Stylable: further information about the completion including type and origin path

hover

Hovers show information about the symbol/object that's below the mouse cursor. This is usually the type of the symbol and a description.

For Stylable: we need to decide on this, but showing the target selector can be really nice, maybe specificity too.

signatureHelp

When the user enters a function or method, display information about the function/method that is being called.

For Stylable: relevant for some of our function syntax - formatters, mixins, states with params

declaration

For the next set of capabilities, we need to determine how the Stylable language maps out in terms of language syntax / capabilities. It is possible we will decide some of these are not relevant to our language.

Resolve the declaration location of a symbol at a given text document position

For Stylable: possibly not relevant to Stylable

definition

Allow the user to see the definition of variables / functions / methods right where the variables / functions / methods are being used.

For Stylable: enables click "go-to" definition behavior

typeDefinition

Resolve the type definition location of a symbol at a given text document position.

For Stylable: probably not relevant to Stylable

implementation

Resolve the implementation location of a symbol at a given text document position.

For Stylable: can be mapped out to the various language features showing implementation for selectors / variables / keyframes

references

Allow the user to see all the source code locations where a certain variable/function/method/symbol is being used.

For Stylable: enables showing all references of a class / element / var / keyframe

documentHighlight

Allow the user to see all occurrences of a symbol in the open editor.

For Stylable: highlight all occurrences of a Stylable symbol, not based on string matching

documentSymbol

Allow the user to quickly navigate to any symbol definition in the open editor.

For Stylable: navigate directly to certain Stylable symbols in the document

workspaceSymbol

Allow the user to quickly navigate to symbol definitions anywhere in the folder (workspace) opened in VS Code.

For Stylable: navigate directly to certain Stylable symbols in the the workspace

codeAction

Provide the user with possible corrective actions right next to an error or warning. If actions are available, a light bulb appears next to the error or warning. When the user clicks the light bulb, a list of available Code Actions is presented.

For Stylable: list of automatic fixes for certain Stylable diagnostics

codeAction resolve

Provide source code manipulation actions such as refactoring. For example, Extract Method.

For Stylable: automatic fixes for certain Stylable diagnostics, or code actions (e.g. extract to variable)

codeLens

Provide the user with actionable, contextual information that is displayed interspersed with the source code.

For Stylable: can be used to show selector specificity or resolved value() inside the source.

codeLens resolve

Bind the CodeLens results to a command by responding to codeLens/resolve.

For Stylable: no immediate useful actions come to mind.

codeLens refresh

Use it to ask clients to refresh the code lenses currently shown in editors

For Stylable: used for updating whatever we decide to display in codeLens if at all

documentLink

Request the location of links in a document

For Stylable: request symbol or url navigation

documentLink resolve

Resolve the target of a given document (if not done in documentLink)

For Stylable: resolve the target for documentLink

documentColor

Allow the user to preview colors in the document

For Stylable: shows color squares next to native colors and variable which resolve to a color

colorPresentation

Allow the user to modify colors in the document

For Stylable: provides a color picker to edit colorPresentation colors

formatting

Provide the user with support for formatting whole documents.

For Stylable: formatting capabilities for *.st.css files

rangeFormatting

Provide the user with support for formatting a selected range of lines in a document.

For Stylable: formatting capabilities for specific ranges in *.st.css files

onTypeFormatting

Provide the user with support for formatting text as they type.

For Stylable: not very important for Stylable

rename

Allow the user to rename a symbol and update all references to the symbol.

For Stylable: rename any Stylable symbol in all occurrences

prepareRename

Setup and test the validity of a rename operation at a given location.

For Stylable: validation for rename actions

foldingRange

Return all folding ranges found in a given text document.

For Stylable: provides collapsing behavior

selectionRange

Return suggested selection ranges at an array of given positions. A selection range is a range around the cursor position which the user might be interested in selecting.

For Stylable: provides contextual expanding selection

semantic tokens

Used to add additional color information to a file that depends on language specific symbol information. A semantic token request usually produces a large result. The protocol therefore supports encoding tokens with numbers. In addition optional support for deltas is available.

For Stylable: provides dynamic syntax highlighting

linkedEditingRange

Return for a given position in a document the range of the symbol at the position and all ranges that have the same content. Optionally a word pattern can be returned to describe valid contents. A rename to one of the ranges can be applied to all other ranges if the new content is valid. If no result-specific word pattern is provided, the word pattern from the client’s language configuration is used.

For Stylable: multi-cursor editing on multiple instance of the same contentreserving comment space

[tomrav]

reserving comment space

[tomrav]

reserving comment space

[tomrav]

reserving comment space