Language Server Protocol support

[gnull]

Hey!

I see that texpresso supports multiple different editors via ad-hoc plugins made separately for each one. Would it make sense to unify the interface and provide texpresso as an LSP?

This way, it would trivially work with many different editors with no need to add support one-by-one. And the LSP interface seems sufficient for texpresso's functionality.

Ivan

[let-def]

Hi,

The existing protocol does the job quite well, and it allows for cheap experimentation with Vim/Emacs. It would not make sense to remove it in my opinion, but adding LSP as an alternative protocol could be a good idea. I don't have the time to do this, but it would be welcome as a contribution.

However, I am thinking that adding support for TeXpresso to an existing LSP server, likely https://github.com/latex-lsp/texlab, is maybe the way to go. TeXpresso can only provide a subset of the features expected from an LSP server.
A dedicated LSP server like TeXlab can make better decision about what features to expose, and when to use TeXpresso and when to use other sources of information.

[lnay]

I just had a play around with a proof of concept for having a language server wrap around the texpresso binary, in order to integrate with Zed which is comparatively much less (not at all) scriptable compared to nvim/emacs:

Screencast.From.2025-04-18.00-50-00.mp4

The hurdle I came across was translating between the row+column representation of text changes, specified in the LSP, with the byte offsets expected by the (change ...) message needed here.
Multibyte characters aside, there's not a convenient way of accessing the offset of the positions in the current state of the buffer (using the protocol). The example above calculates the offsets based on the file on disk (which is why typing at the end of the line behaves the way it does in the clip).

How do you feel about having an extra mode in TeXpresso (on top of "change" and "change-lines") which deals in coordinates? Ideally the column would take into account the sizes of characters in one of UTF-8 or UTF-16 to play well with the LSP.

However, I am thinking that adding support for TeXpresso to an existing LSP server, likely https://github.com/latex-lsp/texlab, is maybe the way to go.

The approach I'm suggesting here is having a language server that just deals with live preview and diagnostics from the build using TeXpresso. And then texlab can be run on the side with the build-related stuff disabled to benefit of its other capabilities (go-to-definition, completions, ...).

[leanhdung1994]

@lnay I found this is related

TeXpresso and the underlying LaTeX compilers are based on a UTF-8 byte representation, and communication between the editor and TeXpresso occurs in terms of byte offsets. However, VS Code only provides access to character positions and not their byte position. Thus, this extension keeps a copy of the current document in a rope data structure (wikipedia), enriched with byte offsets. This allows for efficient conversion between character and byte positions, and also allows for efficient edits to the underlying text string. The code for the rope data structure builds on https://github.com/component/rope.

[lnay]

A prospective language server could also keep track of the full buffer but that feels like a bit unnecessary given that it's already done by both the editor and TeXpresso.

I was curious about one of the maintainers opinion about the complexity of adding this (whether it's done by them or someone else) because the "change-lines" interface already implies that they can get to a given line, and all that's left would be to step across the characters. This extra interface would essentially complete this issue because the language server then becomes just some boilerplate.

[DominikPeters]

I agree that it would be better if TeXpresso handled that; the VS Code extension is a little brittle sometimes which I blame on having to carry a copy around. But someone needs to implement it!

[let-def]

Hi everybody,
Thanks @lnay, that's a cool approach. I just did a quick'n'dirty fork to add support for UTF-16-based offsets: https://github.com/let-def/texpresso/tree/utf-16
The protocol for communicating with the editor features a command (change-range <start-line> <start-char> <end-line> <end-char> "replacement-text") (or ["change-range", ...] in JSON) to update a range using a format for coordinates that should hopefully be compatible with LSP ranges.
I have not tested it because I don't have an LSP-based editor around; I might try to setup Zed with your patch if I have the time.
@DominikPeters would that be enough to solve the problems with the VS Code extension too? Thanks for working around the limitations until now, tell me if there are other things that can help (but I don't promise anything, I really don't have much time for TeXpresso at the moment).

[lnay]

Thanks @let-def, I'll package an MVP later for people to try (atm I have an adhoc-monkeypatched thing going). But I do have a test setup at the moment and a convenient diff from your last commit to see the relevant part of texpresso so I might also open a PR if I find a correct tweak (right now something's not quite working right).

Corrections/clarifications on your command for notes:

• you need the file path just after "change-range"
• you need to include the "-lines" CLI arg

But all this likely after easter, it's been a pleasure starting business 🤝.

[lnay]

I've finished the zed extension. To test the new LSP interface:

• build texpresso with the changes from this branch: https://github.com/let-def/texpresso/tree/utf-16
• Install zed and the extension https://github.com/lnay/zed-texpresso "TeXpresso"
• check the extension README to configure the paths for texpresso and the main tex file

Note that you don't have to explicitly install the language server (this is done automatically with zed's inbuilt node).

Until zed-industries/extensions#2514 gets merged, you need to install it as a "dev extension". Extension build failures are found in the Zed log (search command palette). I have found that the easiest way to fix missing dependencies is to follow the instructions to install dependencies for building Zed edit: it's been merged.

[leanhdung1994]

@lnay I already installed TeXpresso from the main branch. Could you explain how to update it with the changes from your brand? What I do usually is

cd texpresso
(git pull && make)

[let-def]

Thanks :]

[lnay]

changes from your brand

git fetch --all
git switch utf-16

edit: it was merged

[leanhdung1994]

@lnay I run the commands

cd texpresso
git fetch --all
git switch utf-16
(git pull && make)

I doubt that my TeXpresso is updated as intended. Could you have a check on below output?

akira@akira-Zenbook-15-UM3504DA-UM3504DA:~$ cd texpresso
akira@akira-Zenbook-15-UM3504DA-UM3504DA:~/texpresso$ git fetch --all
git switch utf-16
D	test/README.md
D	test/include.tex
M	test/simple.tex
Already on 'utf-16'
Your branch is up to date with 'origin/utf-16'.
akira@akira-Zenbook-15-UM3504DA-UM3504DA:~/texpresso$ (git pull && make)
Already up to date.
make texpresso
make[1]: Entering directory '/home/akira/texpresso'
make -C src texpresso
make[2]: Entering directory '/home/akira/texpresso/src'
make[2]: Nothing to be done for 'texpresso'.
make[2]: Leaving directory '/home/akira/texpresso/src'
make[1]: Leaving directory '/home/akira/texpresso'
make texpresso-tonic
make[1]: Entering directory '/home/akira/texpresso'
make -f Makefile.tectonic tectonic
make[2]: Entering directory '/home/akira/texpresso'
cd tectonic &&  cargo build --release --features external-harfbuzz
warning: tectonic_pdf_io@0.4.1: pdf_io/dpx-mpost.c: In function 'mps_do_page':
warning: tectonic_pdf_io@0.4.1: pdf_io/dpx-mpost.c:1510:3: warning: ignoring return value of 'fread' declared with attribute 'warn_unused_result' [-Wunused-result]
warning: tectonic_pdf_io@0.4.1:  1510 |   fread(buffer, sizeof(char), size, image_file);
warning: tectonic_pdf_io@0.4.1:       |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
warning: unexpected `cfg` condition value: `profile`
  --> crates/engine_xetex/build.rs:28:46
   |
28 |     const PROFILE_BUILD_ENABLED: bool = cfg!(feature = "profile");
   |                                              ^^^^^^^^^^^^^^^^^^^
   |
   = note: expected values for `feature` are: `external-harfbuzz`
   = help: consider adding `profile` as a feature in `Cargo.toml`
   = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
   = note: `#[warn(unexpected_cfgs)]` on by default

warning: `tectonic_engine_xetex` (build script) generated 1 warning
warning: a method with this name may be added to the standard library in the future
   --> crates/bundles/src/cache.rs:376:39
    |
376 |                 if let Err(e) = mfile.lock_shared() {
    |                                       ^^^^^^^^^^^
    |
    = warning: once this associated item is added to the standard library, the ambiguity may cause an error or change in behavior!
    = note: for more information, see issue #48919 <https://github.com/rust-lang/rust/issues/48919>
    = help: call with fully qualified syntax `lock_shared(...)` to keep using the current method
    = note: `#[warn(unstable_name_collisions)]` on by default

warning: `tectonic_bundles` (lib) generated 1 warning
warning: unexpected `cfg` condition name: `has_error_description_deprecated`
   --> src/errors.rs:57:1
    |
57  | / error_chain! {
58  | |     types {
59  | |         Error, ErrorKind, ResultExt, Result;
...   |
104 | | }
    | |_^
    |
    = help: expected names are: `clippy`, `debug_assertions`, `doc`, `docsrs`, `doctest`, `feature`, `fmt_debug`, `miri`, `overflow_checks`, `panic`, `proc_macro`, `relocation_model`, `rustfmt`, `sanitize`, `sanitizer_cfi_generalize_pointers`, `sanitizer_cfi_normalize_integers`, `target_abi`, `target_arch`, `target_endian`, `target_env`, `target_family`, `target_feature`, `target_has_atomic`, `target_has_atomic_equal_alignment`, `target_has_atomic_load_store`, `target_os`, `target_pointer_width`, `target_thread_local`, `target_vendor`, `test`, `ub_checks`, `unix`, and `windows`
    = note: using a cfg inside a macro will use the cfgs from the destination crate and not the ones from the defining crate
    = help: try referring to `impl_error_chain_processed` crate for guidance on how handle this unexpected cfg
    = help: the macro `impl_error_chain_processed` may come from an old version of the `error_chain` crate, try updating your dependency with `cargo update -p error_chain`
    = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
    = note: `#[warn(unexpected_cfgs)]` on by default
    = note: this warning originates in the macro `impl_error_chain_processed` which comes from the expansion of the macro `error_chain` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: unexpected `cfg` condition name: `backtrace`
   --> src/errors.rs:293:11
    |
293 |     #[cfg(backtrace)]
    |           ^^^^^^^^^
    |
    = help: consider using a Cargo feature instead
    = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
             [lints.rust]
             unexpected_cfgs = { level = "warn", check-cfg = ['cfg(backtrace)'] }
    = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(backtrace)");` to the top of the `build.rs`
    = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration

warning: unexpected `cfg` condition name: `backtrace`
   --> src/errors.rs:358:11
    |
358 |     #[cfg(backtrace)]
    |           ^^^^^^^^^
    |
    = help: consider using a Cargo feature instead
    = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
             [lints.rust]
             unexpected_cfgs = { level = "warn", check-cfg = ['cfg(backtrace)'] }
    = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(backtrace)");` to the top of the `build.rs`
    = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration

warning: `tectonic` (lib) generated 3 warnings
    Finished `release` profile [optimized] target(s) in 0.29s
make[2]: Leaving directory '/home/akira/texpresso'
cp -f tectonic/target/release/texpresso-tonic build/
make[1]: Leaving directory '/home/akira/texpresso'
# Build succeeded. Try running:
# build/texpresso test/simple.tex
akira@akira-Zenbook-15-UM3504DA-UM3504DA:~/texpresso$