Skip to content

Latest commit

 

History

History
519 lines (408 loc) · 19.3 KB

File metadata and controls

519 lines (408 loc) · 19.3 KB

Development notes

Coding guidelines

This section contains meta-documentation related to development. For more detailed coding guidelines, please refer to AGENTS.md, which has been organized to be easily recognized by AI agents.

Using local JETLS checkout

During development, you can run JETLS directly from your local checkout using Julia's -m (module run) flag:

julia --startup-file=no --project=/path/to/JETLS -m JETLS serve

This is useful for testing changes without installing the jetls executable. Combined with the JETLS_DEV_MODE preference, this allows you to edit JETLS source code and have Revise pick up changes without restarting the server.

Warning

When using a local checkout other than the release branch (e.g. master), JETLS dependencies may conflict with the dependencies of the code being analyzed. The release branch avoids this by vendoring dependencies with rewritten UUIDs.

[sources] dependencies

In JETLS, since we need to use packages that aren’t yet registered (e.g., JuliaLowering.jl) or specific branches of JET.jl and JuliaSyntax.jl, the Project.toml includes [sources] section. The [sources] section allows simply running Pkg.instantiate() to install all the required versions of these packages on any environment, including the CI setup especially.

On the other hand, it can sometimes be convenient to Pkg.develop some of the packages listed in the [sources] section and edit their source code while developing JETLS. In particular, to have Revise immediately pick up changes made to those packages, we may need to keep them in locally editable directories. However, we cannot run Pkg.develop directly on packages listed in the [sources] section, e.g.:

julia> Pkg.develop("JET")
ERROR: `path` and `url` are conflicting specifications
...

To work around this, you can temporarily comment out the [sources] section and run Pkg.develop("JET"). This lets you use any local JET implementation. After running Pkg.develop("JET"), you can restore the [sources] section, and perform any most of Pkg operations without any issues onward. The same applies to the other packages listed in [sources].

When test fails locally

Some of JETLS's test cases depend on specific implementation details of dependency packages (especially JET and JS/JL), and may fail unless those dependency packages are from the exact commits specified in Project.toml, as mentioned above.

It should be noted that during development, if the versions of those packages already installed in your locally cloned JETLS environment are not updated to the latest ones, you may see some tests fail. In such cases, make sure to run Pkg.update() and re-run the tests.

JETLS_DEV_MODE

JETLS has a development mode that can be enabled through the JETLS_DEV_MODE preference. When this mode is enabled, the language server enables several features to aid in development:

  • Automatic loading of Revise when starting the server, allowing changes to be applied without restarting
  • Uses @invokelatest in message handling to ensure that changes made by Revise are reflected without terminating the runserver loop

Note that error handling behavior (whether errors are caught or propagated) is controlled by JETLS_TEST_MODE, not JETLS_DEV_MODE. See the "JETLS_TEST_MODE" section for details.

You can configure JETLS_DEV_MODE using Preferences.jl:

julia> using Preferences

julia> Preferences.set_preferences!("JETLS", "JETLS_DEV_MODE" => true; force=true) # enable the dev mode

Alternatively, you can directly edit the LocalPreferences.toml file.

While JETLS_DEV_MODE is disabled by default, we strongly recommend enabling it during JETLS development. For development work, we suggest creating the following LocalPreferences.toml file in the root directory of this repository:

LocalPreferences.toml

[JETLS]
JETLS_DEV_MODE = true # enable the dev mode of JETLS

[JET]
JET_DEV_MODE = true # additionally, allow JET to be loaded on nightly

JETLS_TEST_MODE

JETLS has a test mode that controls error handling behavior during testing. When JETLS_TEST_MODE is enabled, the server disables the try/catch error recovery in message handling, ensuring that errors are properly raised during tests rather than being suppressed.

This mode is configured through LocalPreferences.toml and is automatically enabled in the test environment (see test/LocalPreferences.toml).

The error handling behavior in handle_message follows this logic:

  • When !JETLS_TEST_MODE: Errors are caught and logged, allowing the server to continue running
  • When !!JETLS_TEST_MODE: Errors are propagated, ensuring test failures are properly detected

For general users, the server runs with JETLS_TEST_MODE disabled by default, providing error recovery to prevent server crashes during normal use.

Precompilation

JETLS uses precompilation to reduce the latency between server startup and the user receiving first responses. Once you install the JETLS package and precompile it, the language server will start up quickly afterward (until you upgrade the JETLS version), providing significant benefits from the user's perspective.

However, during development, when you're frequently rewriting JETLS code itself, running time-consuming precompilation after each modification might be a waste of time. In such cases, you can disable precompilation by adding the following settings to your LocalPreferences.toml:

LocalPreferences.toml

[JETLS]
precompile_workload = false # Disable precompilation for JETLS

[JET]
precompile_workload = false # Optionally disable precompilation for JET if you're developing it simultaneously

Dynamic registration

This language server supports dynamic registration of LSP features.

With dynamic registration, for example, the server can switch the formatting engine when users change their preferred formatter, or disable specific LSP features upon configuration change, without restarting the server process (although neither of these features has been implemented yet).

Dynamic registration is also convenient for language server development. When enabling LSP features, the server needs to send various capabilities and options to the client during initialization. With dynamic registration, we can rewrite these activation options and re-enable LSP features dynamically, i.e. without restarting the server process.

For example, you can dynamically add , as a triggerCharacter for "completion" as follows. First, launch jetls-client in VSCode1, then add the following diff to unregister the already enabled completion feature. Make a small edit to the file the language server is currently analyzing to send some request from the client to the server. This will allow Revise to apply this diff to the server process via the dev mode callback (see the JETLS.main entrypoint), which should disable the completion feature:

diff --git a/src/completions.jl b/src/completions.jl
index 29d0db5..728da8f 100644
--- a/src/completions.jl
+++ b/src/completions.jl
@@ -21,6 +21,11 @@ completion_options() = CompletionOptions(;
 const COMPLETION_REGISTRATION_ID = "jetls-completion"
 const COMPLETION_REGISTRATION_METHOD = "textDocument/completion"

+let unreg = Unregistration(COMPLETION_REGISTRATION_ID, COMPLETION_REGISTRATION_METHOD)
+    unregister(currently_running, unreg)
+end
+
 function completion_registration()
     (; triggerCharacters, resolveProvider, completionItem) = completion_options()
     documentSelector = DocumentFilter[

Tip

You can add the diff above anywhere Revise can track and apply changes, i.e. any top-level scope in the JETLS module namespace or any subroutine of _handle_message that is reachable upon the request handling.

Warning

Note that currently_running::Server is a global variable that is only defined in JETLS_DEV_MODE. The use of this global variable should be limited to such development purposes and should not be included in normal routines.

After that, delete that diff and add the following diff:

diff --git a/src/completions.jl b/src/completions.jl
index 29d0db5..7609a6a 100644
--- a/src/completions.jl
+++ b/src/completions.jl
@@ -9,6 +9,7 @@ const COMPLETION_TRIGGER_CHARACTERS = [
     "@",  # macro completion
     "\\", # LaTeX completion
     ":",  # emoji completion
+    ",",  # new trigger character
     NUMERIC_CHARACTERS..., # allow these characters to be recognized by `CompletionContext.triggerCharacter`
 ]

@@ -36,6 +37,8 @@ function completion_registration()
             completionItem))
 end

+register(currently_running, completion_registration())
+
 # completion utils
 # ================

This should re-enable completion, and now completion will also be triggered when you type ,.

For these reasons, when adding new LSP features, check whether the feature supports dynamic/static registration, and if it does, actively opt-in to use it. That is, register it via the client/registerCapability request in response to notifications sent from the client, most likely InitializedNotification. The JETLS.register utility is especially useful for this purpose.

Profiling

JETLS provides a mechanism for capturing heap snapshots of the language server process itself. This is useful for investigating JETLS's memory footprint and detecting potential memory leaks in the server implementation.

Taking a heap snapshot

To trigger a heap snapshot, create a .JETLSProfile file in the workspace root directory:

touch .JETLSProfile

When JETLS detects this file, it will:

  1. Take a heap snapshot using Profile.take_heap_snapshot
  2. Save it as JETLS_YYYYMMDD_HHMMSS.heapsnapshot in the workspace root
  3. Show a notification with the file path
  4. Automatically delete the .JETLSProfile trigger file

Analyzing the snapshot

The generated .heapsnapshot file uses the V8 heap snapshot format, which can be analyzed using Chrome DevTools:

  1. Open Chrome and navigate to any page
  2. Open DevTools (F12)
  3. Go to the "Memory" tab
  4. Click "Load" and select the .heapsnapshot file
  5. Use the "Summary" view to see memory usage by type (Constructor)
  6. Use the "Comparison" view to compare two snapshots and identify memory growth

Understanding snapshot metrics

If successful, you should see a view like the following:

Chrome DevTools view

The Summary view shows several columns for each type (Constructor):

  • Distance: Shortest path length from GC roots to the object. Lower distance means more directly reachable from roots.
  • Shallow Size: Memory held directly by the object itself (its own fields), not including memory held by referenced objects.
  • Retained Size: Total memory that would be freed if this object were garbage collected, including all objects exclusively retained by it. This is the most useful metric for finding memory issues.

Using the Retainers panel

When you select an object in the Summary view, the Retainers panel (bottom) shows what is keeping that object alive. This helps trace memory retention back to its root cause. Follow the retention chain upward to understand why objects aren't being garbage collected.

Comparing snapshots

To investigate memory growth:

  1. Take a snapshot shortly after server startup
  2. Perform some operations (open files, trigger analysis, etc.)
  3. Take another snapshot
  4. Compare them in Chrome DevTools using the "Comparison" view

This helps identify which types are accumulating over time.

Command-line analysis

For text-based analysis (useful for sharing or automated processing), use the analyze-heapsnapshot.jl script:

julia --project=scripts scripts/analyze-heapsnapshot.jl JETLS_YYYYMMDD_HHMMSS.heapsnapshot

This displays a summary of memory usage by object type, sorted by shallow size. Use --top=N to control how many entries to show (default: 50).

Limitations

  • Only Julia GC-managed heap is captured; memory allocated by external libraries (BLAS, LAPACK, etc.) is not included
  • The snapshot process itself requires additional memory, so for very large processes, ensure sufficient memory is available

Configuration schema

The configuration schema is generated from the config structs in src/types.jl by two scripts under scripts/schema/:

  • generate.jl generates standalone schema files under schemas/:
    • config-toml.schema.json (complete schema for .JETLSConfig.toml)
    • settings.schema.json (settings only)
    • init-options.schema.json (initialization options only)
  • update-pkg-json.jl updates VSCode configuration properties inside jetls-client/package.json (with $defs inlined and description renamed to markdownDescription).

When you modify the config structs, update schemas/description.toml with descriptions for any new or changed fields, then regenerate all schemas by running:

./scripts/schema/regenerate.sh

Both scripts support a --check flag that exits with an error if the output is out of date. CI runs them in this mode to ensure the generated files are kept in sync.

Release process

JETLS avoids dependency conflicts with packages being analyzed by rewriting the UUIDs of its dependencies and vendoring them. This allows JETLS to have its own isolated copies of dependencies that won't conflict with the packages users are analyzing.

Branch strategy

  • master: Development branch where regular development happens. Dependencies keep their original UUIDs.
  • release: Distribution branch for users. Dependencies are vendored with rewritten UUIDs and the vendor/ directory contains copies of all dependency packages.
  • releases/YYYY-MM-DD: Release preparation branches. These branches are created from release, merged with master, vendored, and then merged back into release via pull requests on GitHub. These branches can be deleted after merging because the [sources] entries reference specific commit SHAs, not branch names.

Pinned installation

Users can install a specific release version using Julia's package manager with the release tag:

julia -e 'using Pkg; Pkg.Apps.add(; url="https://github.com/aviatesk/JETLS.jl", rev="YYYY-MM-DD")'

Replace YYYY-MM-DD with the desired release date (e.g., 2025-11-27).

Release procedure

The release process is automated via scripts/prepare-release.sh:

./scripts/prepare-release.sh YYYY-MM-DD

This script performs all the steps below automatically and creates a pull request. After the script completes:

  1. Merge the pull request using "Create a merge commit" (not squash or rebase). This preserves the merge history from master, allowing you to track which master commits are included in release. The CI will run tests on the vendored environment before merging.

  2. The releases/YYYY-MM-DD branch can be deleted after merging. The [sources] entries reference specific commit SHAs, so the branch is no longer needed.

After the PR is merged, CHANGELOG.md on master will be automatically updated by the CI workflow.

Manual steps

The prepare-release.sh script automates the following steps:

  1. Create a release branch from release and merge master

    git checkout release
    git pull origin release
    JETLS_VERSION='YYYY-MM-DD'
    git checkout -b releases/$JETLS_VERSION
    git merge master -X theirs
  2. Vendor dependency packages with local paths

    julia --startup-file=no --project=. scripts/vendor-deps.jl --source-branch=master --local

    This generates vendor/ with local path references in [sources].

  3. Commit and push the vendor directory

    git add -A
    git commit -m "release: update vendored dependencies ($JETLS_VERSION)"
    git push -u origin releases/$JETLS_VERSION
  4. Update [sources] to reference the vendor commit SHA

    VENDOR_COMMIT=$(git rev-parse HEAD)
    julia --startup-file=no --project=. scripts/vendor-deps.jl --source-branch=master --rev="$VENDOR_COMMIT"

    This updates [sources] entries to use the commit SHA instead of local paths.

  5. Commit and push the release

    echo "$JETLS_VERSION" > JETLS_VERSION
    git add -A
    git commit -m "release: $JETLS_VERSION"
    git push origin releases/$JETLS_VERSION

    Important: The commit message must follow the release: YYYY-MM-DD format exactly. The documentation CI extracts this date to display in the release documentation's index page.

  6. Create a pull request from releases/YYYY-MM-DD to release.

Note

vendor-deps.jl generates UUIDs using uuid5(original_uuid, "JETLS-vendor"). This is deterministic, so the same vendored UUID is always generated for the same original UUID, ensuring consistency across multiple vendoring operations

Note

To check which master commits are not yet included in release:

git log master ^release --oneline

Local testing of vendored environment

To test the vendored environment locally without committing:

julia --startup-file=no --project=. scripts/vendor-deps.jl --source-branch=master --local

The --local flag uses local path references in [sources] instead of GitHub URLs, allowing you to test with the locally generated vendor/ directory.

jetls-client development

The jetls-client directory contains the VSCode language client extension for JETLS.

Development setup

  1. Navigate to the jetls-client directory:

    cd jetls-client
  2. Install dependencies:

    npm install
  3. Build the extension:

    npm run build

    Or for development with watch mode:

    npm run build:watch

Testing the extension locally

To test the extension locally in VSCode:

  1. Open the jetls-client directory in VSCode
  2. Press F5 to launch the Extension Development Host
  3. The extension will be loaded in the new VSCode window

To use a local JETLS.jl checkout with the development extension (see Using local JETLS checkout), configure jetls-client.executable in your settings.json using the array form:

{
  "jetls-client.executable": [
    "julia",
    "--startup-file=no",
    "--history-file=no",
    "--project=/path/to/JETLS",
    "-m",
    "JETLS",
    "serve"
  ]
}

Publishing

To publish the extension to the marketplace:

cd jetls-client
vsce publish [patch|minor|major] -m "jetls-client: vX.Y.Z"

This will bump the version in package.json, create a git commit and tag with the specified message, and publish to the marketplace in one step.

Footnotes

  1. Of course, the hack explained here is only possible with clients that support dynamic registration. VSCode is currently one of the frontends that best supports dynamic registration.