comm: complete migration to JSON.jl v1 by aviatesk · Pull Request #301 · aviatesk/JETLS.jl

aviatesk · 2025-11-17T17:49:24Z

This commit completes the migration to JSON.jl v1. As explained in
#300, JSON.jl v1 no longer supports out-of-box
deserialization for custom data types and now requires explicit
tags for each custom data type field (see
https://publish.obsidian.md/jetls/work/JETLS/Switch+JSON3.jl+to+JSON.jl+v1#2025-11-16).

This presents a significant challenge for LSP.jl, which uses the
@interface DSL for TypeScript-like type definitions, because
such tag generation must also be done through macro generation.
Without this approach, we would need to scatter @tags macros
and their associated choosetype definitions throughout the code
base, which should be very very tedious.

This commit further complicates the already complex @interface
implementation to implement such automatic tag generation.
While this macro has arguably become one of the most arcane
Julia macros in existence, it does function correctly.
Numerous tests have also been added.

Most importantly, completing the switch to JSON v1 provides the
following JSON communication performance improvements:

using LSP

uri = LSP.URIs2.filepath2uri(abspath("src/JETLS.jl"))
s = """{
  "jsonrpc": "2.0",
  "id": 0,
  "method": "textDocument/completion",
  "params": {
    "textDocument": {
      "uri": "$uri"
    },
    "position": {
      "line": 0,
      "character": 0
    },
    "workDoneToken": "workDoneToken",
    "partialResultToken": "partialResultToken"
  }
}""";

@time x = LSP.to_lsp_object(s)
@time LSP.to_lsp_json(x)

using BenchmarkTools
@benchmark LSP.to_lsp_object($s)
@benchmark LSP.to_lsp_json(x) setup=(x=LSP.to_lsp_object(s))

master

julia> @time x = LSP.to_lsp_object(s)
  0.010983 seconds (328 allocations: 17.172 KiB, 90.51% compilation time)
CompletionRequest("2.0", 0, "textDocument/completion", CompletionParams(TextDocumentIdentifier(file:///Users/aviatesk/julia/packages/JETLS/src/JETLS.jl), Position(0x0000000000000000, 0x0000000000000000), "workDoneToken", "partialResultToken", nothing))

julia> @time LSP.to_lsp_json(x)
  0.000086 seconds (54 allocations: 3.500 KiB)
"{\"jsonrpc\":\"2.0\",\"id\":0,\"method\":\"textDocument/completion\",\"params\":{\"textDocument\":{\"uri\":\"file:///Users/aviatesk/julia/packages/JETLS/src/JETLS.jl\"},\"position\":{\"line\":0,\"character\":0},\"workDoneToken\":\"workDoneToken\",\"partialResultToken\":\"partialResultToken\"}}"

julia> using BenchmarkTools

julia> @benchmark LSP.to_lsp_object($s)
BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):  357.500 μs … 537.833 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     370.291 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   375.247 μs ±  17.525 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

     █▆▂▃▇▅▃
  ▁▃▇████████▆▅▅▅▄▄▃▃▂▂▂▂▂▂▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
  358 μs           Histogram: frequency by time          453 μs <

 Memory estimate: 5.91 KiB, allocs estimate: 120.

julia> @benchmark LSP.to_lsp_json(x) setup=(x=LSP.to_lsp_object(s))
BenchmarkTools.Trial: 10000 samples with 10 evaluations per sample.
 Range (min … max):  1.871 μs … 862.158 μs  ┊ GC (min … max): 0.00% … 98.95%
 Time  (median):     2.104 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   2.379 μs ±  10.872 μs  ┊ GC (mean ± σ):  6.36% ±  1.40%

    ▄▇█▆▄▄▂▁
  ▂▇█████████▇▅▅▄▄▄▃▄▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃
  1.87 μs         Histogram: frequency by time        3.61 μs <

 Memory estimate: 2.70 KiB, allocs estimate: 36.

this PR

julia> @time x = LSP.to_lsp_object(s)
  0.010483 seconds (304 allocations: 15.422 KiB, 96.27% compilation time)
CompletionRequest("2.0", 0, "textDocument/completion", CompletionParams(TextDocumentIdentifier(file:///Users/aviatesk/worktrees/JETLS/JSON-v1-again/src/JETLS.jl), Position(0x0000000000000000, 0x0000000000000000), "workDoneToken", "partialResultToken", nothing))

julia> @time LSP.to_lsp_json(x)
  0.000095 seconds (69 allocations: 3.969 KiB)
"{\"jsonrpc\":\"2.0\",\"id\":0,\"method\":\"textDocument/completion\",\"params\":{\"textDocument\":{\"uri\":\"file:///Users/aviatesk/worktrees/JETLS/JSON-v1-again/src/JETLS.jl\"},\"position\":{\"line\":0,\"character\":0},\"workDoneToken\":\"workDoneToken\",\"partialResultToken\":\"partialResultToken\"}}"

julia> using BenchmarkTools

julia> @benchmark LSP.to_lsp_object($s)
BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):  14.875 μs … 104.000 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     15.333 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   15.612 μs ±   2.103 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▁▄▆█▇▇▆▆▄▃▂▁                                                 ▂
  █████████████▇▆▆▆▆▆▇▆▅▃▅▆▆▅▆▆▅▅▅▄▂▆▇▆▇▇█▆▅▆▆▆▆▅▆▅▆▅▅▄▄▄▃▂▄▂▃ █
  14.9 μs       Histogram: log(frequency) by time      20.5 μs <

 Memory estimate: 4.45 KiB, allocs estimate: 83.

julia> @benchmark LSP.to_lsp_json(x) setup=(x=LSP.to_lsp_object(s))
BenchmarkTools.Trial: 10000 samples with 9 evaluations per sample.
 Range (min … max):  2.181 μs …  1.146 ms  ┊ GC (min … max): 0.00% … 99.38%
 Time  (median):     2.361 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   2.535 μs ± 11.437 μs  ┊ GC (mean ± σ):  4.49% ±  0.99%

     ▃▄█▅▇▃▄▁▁
  ▂▄▆█████████▇▇▅▅▄▄▃▃▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃
  2.18 μs        Histogram: frequency by time        3.39 μs <

 Memory estimate: 2.48 KiB, allocs estimate: 34.

codecov · 2025-11-17T17:51:28Z

Codecov Report

❌ Patch coverage is 31.81818% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.09%. Comparing base (7fc4ccb) to head (10c1bb7).

Files with missing lines	Patch %	Lines
LSP/src/DSL/interface.jl	7.40%	25 Missing ⚠️
LSP/src/communication.jl	75.00%	2 Missing ⚠️
src/testrunner/testrunner.jl	0.00%	2 Missing ⚠️
LSP/src/LSP.jl	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #301      +/-   ##
==========================================
+ Coverage   64.63%   65.09%   +0.46%     
==========================================
  Files          62       63       +1     
  Lines        8154     8097      -57     
==========================================
+ Hits         5270     5271       +1     
+ Misses       2884     2826      -58

Flag	Coverage Δ
JETLS.jl	`66.44% <0.00%> (ø)`
LSP.jl	`31.29% <33.33%> (+5.13%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

aviatesk · 2025-11-19T21:01:52Z

/gemini review

aviatesk · 2025-11-19T21:12:11Z

@gemini-code-assist Where are the specific comments...

aviatesk · 2025-11-19T21:20:05Z

@gemini-code-assist You didn't post any comments and that's why I'm asking for it.

@time

This commit completes the migration to JSON.jl v1. As explained in #300, JSON.jl v1 no longer supports out-of-box deserialization for custom data types and now requires explicit tags for each custom data type field (see https://publish.obsidian.md/jetls/work/JETLS/Switch+JSON3.jl+to+JSON.jl+v1#2025-11-16). This presents a significant challenge for LSP.jl, which uses the `@interface` DSL for TypeScript-like type definitions, because such tag generation must also be done through macro generation. Without this approach, we would need to scatter `@tags` macros and their associated `choosetype` definitions throughout the code base, which should be very very tedious. This commit further complicates the already complex `@interface` implementation to implement such automatic tag generation. While this macro has arguably become one of the most arcane Julia macros in existence, it does function correctly. Numerous tests have also been added. Most importantly, completing the switch to JSON v1 provides the following JSON communication performance improvements: ```julia using LSP uri = LSP.URIs2.filepath2uri(abspath("src/JETLS.jl")) s = """{ "jsonrpc": "2.0", "id": 0, "method": "textDocument/completion", "params": { "textDocument": { "uri": "$uri" }, "position": { "line": 0, "character": 0 }, "workDoneToken": "workDoneToken", "partialResultToken": "partialResultToken" } }"""; @time x = LSP.to_lsp_object(s) @time LSP.to_lsp_json(x) using BenchmarkTools @benchmark LSP.to_lsp_object($s) @benchmark LSP.to_lsp_json(x) setup=(x=LSP.to_lsp_object(s)) ``` > master ```julia-repl julia> @time x = LSP.to_lsp_object(s) 0.010983 seconds (328 allocations: 17.172 KiB, 90.51% compilation time) CompletionRequest("2.0", 0, "textDocument/completion", CompletionParams(TextDocumentIdentifier(file:///Users/aviatesk/julia/packages/JETLS/src/JETLS.jl), Position(0x0000000000000000, 0x0000000000000000), "workDoneToken", "partialResultToken", nothing)) julia> @time LSP.to_lsp_json(x) 0.000086 seconds (54 allocations: 3.500 KiB) "{\"jsonrpc\":\"2.0\",\"id\":0,\"method\":\"textDocument/completion\",\"params\":{\"textDocument\":{\"uri\":\"file:///Users/aviatesk/julia/packages/JETLS/src/JETLS.jl\"},\"position\":{\"line\":0,\"character\":0},\"workDoneToken\":\"workDoneToken\",\"partialResultToken\":\"partialResultToken\"}}" julia> using BenchmarkTools julia> @benchmark LSP.to_lsp_object($s) BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample. Range (min … max): 357.500 μs … 537.833 μs ┊ GC (min … max): 0.00% … 0.00% Time (median): 370.291 μs ┊ GC (median): 0.00% Time (mean ± σ): 375.247 μs ± 17.525 μs ┊ GC (mean ± σ): 0.00% ± 0.00% █▆▂▃▇▅▃ ▁▃▇████████▆▅▅▅▄▄▃▃▂▂▂▂▂▂▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂ 358 μs Histogram: frequency by time 453 μs < Memory estimate: 5.91 KiB, allocs estimate: 120. julia> @benchmark LSP.to_lsp_json(x) setup=(x=LSP.to_lsp_object(s)) BenchmarkTools.Trial: 10000 samples with 10 evaluations per sample. Range (min … max): 1.871 μs … 862.158 μs ┊ GC (min … max): 0.00% … 98.95% Time (median): 2.104 μs ┊ GC (median): 0.00% Time (mean ± σ): 2.379 μs ± 10.872 μs ┊ GC (mean ± σ): 6.36% ± 1.40% ▄▇█▆▄▄▂▁ ▂▇█████████▇▅▅▄▄▄▃▄▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃ 1.87 μs Histogram: frequency by time 3.61 μs < Memory estimate: 2.70 KiB, allocs estimate: 36. ``` > this PR ```julia-repl julia> @time x = LSP.to_lsp_object(s) 0.010483 seconds (304 allocations: 15.422 KiB, 96.27% compilation time) CompletionRequest("2.0", 0, "textDocument/completion", CompletionParams(TextDocumentIdentifier(file:///Users/aviatesk/worktrees/JETLS/JSON-v1-again/src/JETLS.jl), Position(0x0000000000000000, 0x0000000000000000), "workDoneToken", "partialResultToken", nothing)) julia> @time LSP.to_lsp_json(x) 0.000095 seconds (69 allocations: 3.969 KiB) "{\"jsonrpc\":\"2.0\",\"id\":0,\"method\":\"textDocument/completion\",\"params\":{\"textDocument\":{\"uri\":\"file:///Users/aviatesk/worktrees/JETLS/JSON-v1-again/src/JETLS.jl\"},\"position\":{\"line\":0,\"character\":0},\"workDoneToken\":\"workDoneToken\",\"partialResultToken\":\"partialResultToken\"}}" julia> using BenchmarkTools julia> @benchmark LSP.to_lsp_object($s) BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample. Range (min … max): 14.875 μs … 104.000 μs ┊ GC (min … max): 0.00% … 0.00% Time (median): 15.333 μs ┊ GC (median): 0.00% Time (mean ± σ): 15.612 μs ± 2.103 μs ┊ GC (mean ± σ): 0.00% ± 0.00% ▁▄▆█▇▇▆▆▄▃▂▁ ▂ █████████████▇▆▆▆▆▆▇▆▅▃▅▆▆▅▆▆▅▅▅▄▂▆▇▆▇▇█▆▅▆▆▆▆▅▆▅▆▅▅▄▄▄▃▂▄▂▃ █ 14.9 μs Histogram: log(frequency) by time 20.5 μs < Memory estimate: 4.45 KiB, allocs estimate: 83. julia> @benchmark LSP.to_lsp_json(x) setup=(x=LSP.to_lsp_object(s)) BenchmarkTools.Trial: 10000 samples with 9 evaluations per sample. Range (min … max): 2.181 μs … 1.146 ms ┊ GC (min … max): 0.00% … 99.38% Time (median): 2.361 μs ┊ GC (median): 0.00% Time (mean ± σ): 2.535 μs ± 11.437 μs ┊ GC (mean ± σ): 4.49% ± 0.99% ▃▄█▅▇▃▄▁▁ ▂▄▆█████████▇▇▅▅▄▄▃▃▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃ 2.18 μs Histogram: frequency by time 3.39 μs < Memory estimate: 2.48 KiB, allocs estimate: 34. ```

aviatesk force-pushed the avi/JSON-v1-again branch 12 times, most recently from f6ee6eb to 6f060b1 Compare November 19, 2025 20:02

This comment was marked as off-topic.

Sign in to view

This comment was marked as spam.

Sign in to view

aviatesk mentioned this pull request Nov 20, 2025

LSP: a bunch of code cleanups/refactoring #309

Merged

aviatesk force-pushed the avi/JSON-v1-again branch 2 times, most recently from 7ee747b to 7c9eaca Compare November 20, 2025 15:40

aviatesk mentioned this pull request Nov 20, 2025

LSP: add precompiles and tests #310

Merged

aviatesk force-pushed the avi/JSON-v1-again branch 3 times, most recently from 67b766f to 5e4022d Compare November 21, 2025 07:14

aviatesk force-pushed the master branch from 9ab937d to 0a65a5a Compare November 22, 2025 17:00

aviatesk force-pushed the avi/JSON-v1-again branch 3 times, most recently from 9ba579a to 0046553 Compare November 23, 2025 16:46

aviatesk force-pushed the avi/JSON-v1-again branch 5 times, most recently from be6dc80 to 0dfb05b Compare November 24, 2025 21:10

aviatesk force-pushed the avi/JSON-v1-again branch from 0dfb05b to 77f1a62 Compare February 9, 2026 14:37

aviatesk force-pushed the avi/JSON-v1-again branch from 77f1a62 to 10c1bb7 Compare February 9, 2026 14:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

comm: complete migration to JSON.jl v1#301

comm: complete migration to JSON.jl v1#301
aviatesk wants to merge 1 commit intomasterfrom
avi/JSON-v1-again

aviatesk commented Nov 17, 2025 •

edited

Loading

Uh oh!

codecov bot commented Nov 17, 2025 •

edited

Loading

Uh oh!

aviatesk commented Nov 19, 2025

Uh oh!

This comment was marked as off-topic.

Uh oh!

aviatesk commented Nov 19, 2025

Uh oh!

This comment was marked as spam.

aviatesk commented Nov 19, 2025

Uh oh!

This comment was marked as spam.

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

aviatesk commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

aviatesk commented Nov 19, 2025

Uh oh!

This comment was marked as off-topic.

Uh oh!

aviatesk commented Nov 19, 2025

Uh oh!

This comment was marked as spam.

aviatesk commented Nov 19, 2025

Uh oh!

This comment was marked as spam.

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

aviatesk commented Nov 17, 2025 •

edited

Loading

codecov bot commented Nov 17, 2025 •

edited

Loading