The Benefits of Redesigning Apache Lucene.NET Documentation with MyST Markdown

[jbampton]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem? Please describe the problem.

Benefits of Redesigning Apache Lucene.NET Documentation with MyST Markdown

Redesigning the Apache Lucene.NET documentation site with MyST Markdown offers the project modernized authoring, native math/code rendering, and streamlined publishing. It enables highly interactive technical documentation and smooth synchronization with modern developer tools.

Key Benefits for Apache Lucene.NET

• Advanced Technical Authoring: MyST Markdown extends standard Markdown to natively support complex technical content. This includes advanced data tables, footnotes, and inline/block mathematical formulas (highly useful for search scoring and Okapi BM25 algorithms).
• Highly Interactive Experiences: MyST compiles content into an Abstract Syntax Tree (AST). This allows it to translate docs into interactive React-based websites, PDFs, or Jupyter Notebook-style computational narratives.
• Superior Cross-Referencing: Built-in capabilities for scientific citations, cross-referencing, and rich-preview structured links make it much easier for developers to navigate complex API dependencies, architecture diagrams, and search methodologies.
• Sphinx & Docs-as-Code Compatibility: MyST brings the powerful tooling and extensibility of the Sphinx ecosystem to standard Markdown. This allows the Lucene.NET community to maintain clean, readable documentation without sacrificing structural power.
• Open Technical Community: By adopting a format built by the Executable Book Project, the team taps into a massive open-source ecosystem. This simplifies the authoring and maintenance processes for contributors.

---

https://mystmd.org/guide/quickstart

https://github.com/jupyter-book

---

So I just went and tested a localhost hot reloading server with the mystmd CLI. So to change to full MyST MD some of the navigation, linking, TOCs would configuring perhaps more. I have never documented C# before but I have recently built two MyST sites. You can boot up an instant live server with uvx from the repo root it handles all the dependencies and basically with two commands you can boot up a MyST site:

First run:

uvx --from mystmd myst init 

Then run:

uvx --from mystmd myst start

The first command will pause and ask a question just enter yes:

Describe the solution you'd like

A new modern collaborator friendly site

Additional context

[paulirwin]

@jbampton Thanks for the feature request. I also have had the idea to improve and overhaul our docs; I feel they're a bit stale. But, you see those xref: things in your screenshots... that's one problem. docfx that we use currently can resolve those to actual .NET types/members/etc and create the correct cross-links to those docs.

Also note that there's a lot more to the docs than just the markdown files. It also extracts the docs from the XML doc comments on the types themselves. Example: https://lucenenet.apache.org/docs/4.8.0-beta00018/api/core/Lucene.Net.Index.IndexWriter.html

Without xref support and XML doc comments support, I feel like this is a non-starter. However, we might be able to adopt some parts of this (more modern styles, improved search, better rendering, etc) within our existing docfx-based infrastructure. Or, we could contribute upstream to docfx what is missing from what we want.

For now, I'm moving this to the Future milestone for consideration as I don't think it can be accomplished for 4.8.0, but I like the styles and ideas here and don't want to close it outright.