Setup Proper Documentation (jupyterlab#69)

* fix-readme

* include-flags

* fix-lint

* fix-test

* look-for-home-in-env-too

* set-cli-flags-in-advance-section

Author: Darshan808

README.md

@@ -5,104 +5,132 @@
 [![version on PyPI](https://img.shields.io/pypi/v/jupyter-builder.svg)](https://pypi.org/project/jupyter-builder/)
 [![version on conda-forge](https://img.shields.io/conda/vn/conda-forge/jupyter-builder.svg)](https://anaconda.org/conda-forge/jupyter-builder)
 
-Build tools for JupyterLab (and remixes)
+Build tools for JupyterLab extensions — extracted from the core [JupyterLab](https://github.com/jupyterlab/jupyterlab) codebase to be maintained and used independently.
 
-> \[!NOTE\]
-> This started by the extraction of the builder tools included in
-> the core [JupyterLab](https://github.com/jupyterlab/jupyterlab) during a GSoC project. See [below](#gsoc-2024-anecdote) for more information.
+## Installation
 
-## Why extracting the build tools?
-
-- This would also solve some chicken-and-egg problems like jupyterlab/jupyterlab_pygments#23.
-- Isolating the builder functionalities will simplify the work
-  of core and extension developers who can now focus on their respective parts of the
-  codebase instead of the earlier intertwined code.
-- It will in particular reduce the need to update the maintenance tooling to produce extension compatible with newer version of Jupyter app.
-
-## How to install the package?
-
-Execute the following command in a terminal:
-
-```
+```bash
 pip install jupyter_builder
 ```
 
-## What does it do?
-
-- Provides a CLI for building Jupyter extensions. There are 3 subcommands
-  - `build` : Builds the Jupyter extension JavaScript assets to be consumed by the Jupyter app.
-    ```
-    jupyter-builder build <path to extension folder>
-    ```
-  - `develop` : Install the Jupyter extension JavaScript assets in dev mode for consumption in the Jupyter app. It similar to [editable install mode of pip](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs)
-    ```
-    jupyter-builder develop --overwrite <path to extension folder>
-    ```
-  - `watch` : Automatically rebuild the development JavaScript assets when one file is changed to ease development.
-    ```
-    jupyter-builder watch <path to extension folder>
-    ```
-- Provides a NPM package manager: `jlpm`
+## CLI
 
-## How to uninstall the package?
+### `build`
 
-Execute the following command in a terminal:
+Compile the extension JavaScript assets for consumption by a Jupyter app.
 
-```
-pip uninstall jupyter_builder
+```bash
+jupyter-builder build <path/to/extension>
 ```
 
-______________________________________________________________________
+### `develop`
 
-## GSoC 2024 Anecdote
+Install extension assets in development mode (analogous to `pip install -e`). Uses a symlink by default.
 
-> Written by [@cronan03](https://github.com/cronan03) - GSoC Contributor 2024
+```bash
+jupyter-builder develop <path/to/extension>
+```
 
-### Goal
+### `watch`
 
-The goals of this project are:
+Automatically rebuild development assets when source files change.
 
-- to extract that tooling as a new separate package to ease maintenance (for core and extension developers)
-- update the existing package to use the new one
-- make it configurable to be reused for other applications.
+```bash
+jupyter-builder watch <path/to/extension>
+```
 
-### What I did
+> For advanced configuration, see the [Advanced](#advanced) section for available flags.
 
-1. Successfully created a CLI with the processes `develop`, `build` and `watch` mentioned above.
-1. Created extensive unit tests using `Pytest` to ensure the processes run efficiently on any OS.
-1. Reduced external dependencies by bringing `jlpm` and `jupyterlab.semver` to the package.
-1. Pre released the package. It can be found on Pypi here https://pypi.org/project/jupyter-builder/
-1. Initiated a solution to the issue https://github.com/jupyterlab/jupyterlab/issues/13456
+## `jlpm`
 
-### What's left to do
+`jupyter-builder` ships `jlpm`, a Jupyter-aware Node.js package manager wrapper:
 
-1. We should bring `@jupyterlab/builder` within this package and make it generic.
-   For now the code lives there: https://github.com/jupyterlab/jupyterlab/tree/main/builder
-1. https://github.com/jupyterlab/jupyter-builder/blob/fffb100fc57ecb147bface4441f91bfd0cb6ff9a/jupyter_builder/federated_extensions.py#L296 which is responsible for checking version overlap has been temporarily ignored to make the build feature work.
-1. Update JupyterLab core to use this new package
+```bash
+jlpm install
+jlpm build
+```
 
-### Merged PRs
+## Advanced
+
+### CLI flags
+
+<details>
+<summary><code>build</code></summary>
+
+| Flag                         | Description                                                            |
+| ---------------------------- | ---------------------------------------------------------------------- |
+| `--development`              | Build in development mode (default: `False`)                           |
+| `--source-map`               | Generate source maps (default: `False`)                                |
+| `--static-url=<url>`         | Set the URL for static assets                                          |
+| `--core-version=<version>`   | JupyterLab core version to build against                               |
+| `--core-package-file=<path>` | Path to a core application `package.json` (overrides `--core-version`) |
+
+</details>
+
+<details>
+<summary><code>develop</code></summary>
+
+| Flag                         | Description                                  |
+| ---------------------------- | -------------------------------------------- |
+| `--overwrite`                | Overwrite existing files                     |
+| `--user`                     | Install to the user's directory              |
+| `--sys-prefix`               | Install under `sys.prefix` (default: `True`) |
+| `--labextensions-dir=<path>` | Install to a custom labextensions directory  |
+
+</details>
+
+<details>
+<summary><code>watch</code></summary>
+
+| Flag                         | Description                                                            |
+| ---------------------------- | ---------------------------------------------------------------------- |
+| `--development`              | Build in development mode (default: `True`)                            |
+| `--source-map`               | Generate source maps (default: `False`)                                |
+| `--core-version=<version>`   | JupyterLab core version to build against                               |
+| `--core-package-file=<path>` | Path to a core application `package.json` (overrides `--core-version`) |
+
+</details>
+
+### Python API
+
+```python
+from jupyter_builder.federated_extensions import (
+    build_labextension,
+    develop_labextension_py,
+    watch_labextension,
+)
+
+build_labextension(
+    "/path/to/extension",
+    development=False,
+    source_map=False,
+    static_url=None,
+    core_version=None,
+    core_package_file=None,
+)
+
+develop_labextension_py(
+    "/path/to/extension",
+    overwrite=True,
+    symlink=True,
+    user=False,
+    sys_prefix=True,
+)
+
+watch_labextension(
+    "/path/to/extension",
+    labextensions_path=[...],
+    development=True,
+    source_map=False,
+)
+```
 
-1. https://github.com/jupyterlab/jupyter-builder/pull/11
-   - This PR focuses on extracting the `develop` feature which is responsible for installing the Jupyter extension JS assets in dev mode.
-   - Considering the size of [labextension.py](https://github.com/jupyterlab/jupyterlab/blob/main/jupyterlab/labextensions.py), only features essential to Jupyter builder were added.
-   - Each of the features will inherit from the class `BaseExtensionApp` present [here](https://github.com/jupyterlab/jupyter-builder/blob/main/jupyter_builder/base_extension_app.py)
-   - The [federated_extensions.py](https://github.com/jupyterlab/jupyter-builder/blob/main/jupyter_builder/federated_extensions.py)  sets up and executes commands to build, develop and waatch a JupyterLab extension. It resolves paths, constructs the appropriate command-line arguments, and executes the build process using `subprocess.check_call`. Optional parameters allow for customization of the build process, including logging, development mode, and source map generation.
-1. https://github.com/jupyterlab/jupyter-builder/pull/13
-   - This PR focuses on extracting the `build` feature which is responsible for creating the Javascript assets which will be consumed by the Jupyter App.
-   - It will always result in the creation of a file `static/style.js` in `<extension_folder>/myextension/labextension`.
-   - Tests have been crafted using `Pytest` to check for the existence of files mentioned above on running the `build` command.
-1. https://github.com/jupyterlab/jupyter-builder/pull/18
-   - The `watch` feature on running will rebuild the JS assets on being triggered. This happens on changing contents in `<extension_folder>/src/index.ts`
-   - To test this feature we deliberately make a change in `index.ts` triggering `watch`. This replaces old JS assets with new ones having different hash values in the file names. We create 2 vectors of filenames before and after triggering `watch` which will tell us if it actually worked.
-1. https://github.com/jupyterlab/jupyter-builder/pull/20
-   - To reduce external dependencies, we added `jlpm` to this package.
-   - It existed [here](https://github.com/jupyterlab/jupyterlab/blob/main/jupyterlab/jlpmapp.py) with the [entrypoint](https://github.com/jupyterlab/jupyterlab/blob/e048f27548969c0e4403417ac04bc186f119128f/pyproject.toml#L60).
-1. https://github.com/jupyterlab/jupyter-builder/pull/22
-   - Documented the working of the Jupyter builder along with installation guide.
+## Uninstall
+
+```bash
+pip uninstall jupyter_builder
+```
 
-### Challenges and Learning
+## Credits
 
-1. One of the main challenges was starting this project from scratch with no pre existing code to rely on. I thank my mentor [@fcollonval](https://github.com/fcollonval) for creating the skeleton in https://github.com/jupyterlab/jupyter-builder/pull/2 which gave me a base to work on.
-1. Selecting relevant features for Jupyter builder from [labextension.py](https://github.com/jupyterlab/jupyterlab/blob/main/jupyterlab/labextensions.py) was really tough without a thorough understanding on which functions Jupyter builder would actually rely on.
-1. Creating tests for the `watch` feature was tricky as I had to carefully adjust sleep times to make sure the function was running before a change in `<extension_folder>/src/index.ts` was made. Otherwise the change happened before `watch` ran and never triggered it.
+This package was initially created during [GSoC 2024](https://summerofcode.withgoogle.com) by [@cronan03](https://github.com/cronan03), mentored by [@fcollonval](https://github.com/fcollonval).

jupyter_builder/core_path.py

@@ -10,6 +10,11 @@
 import requests
 
 
+def _home_dir() -> Path:
+    home = os.environ.get("HOME")
+    return Path(home) if home else Path.home()
+
+
 def get_core_meta(
     version: Optional[str] = None, ext_path: Optional[Union[str, os.PathLike[str]]] = None
 ) -> str:
@@ -22,7 +27,7 @@ def get_core_meta(
                 return installed_core_meta
         requested_version = "main"
 
-    cache_root = Path.home() / ".cache" / "jupyterlab_builder" / "core"
+    cache_root = _home_dir() / ".cache" / "jupyterlab_builder" / "core"
     cached_file = _get_cached_core_meta_file(cache_root, requested_version)
     if cached_file is not None:
         return str(cached_file)

tests/test_core_path.py

@@ -291,7 +291,9 @@ def test_ensure_builder_reads_custom_core_package_file(tmp_path):
     ext_path = tmp_path / "ext"
     core_path_dir = tmp_path / "core-meta"
     core_package_file = core_path_dir / "core.package.json"
+    # TODO: Once core starts using new @jupyter/builder, we need to update here.
     builder_dir = ext_path / "node_modules" / "@jupyterlab" / "builder"
+    new_builder_dir = ext_path / "node_modules" / "@jupyter" / "builder"
 
     builder_dir.mkdir(parents=True)
     core_path_dir.mkdir()
@@ -304,4 +306,4 @@ def test_ensure_builder_reads_custom_core_package_file(tmp_path):
 
     builder_path = _ensure_builder(str(ext_path), str(core_package_file))
 
-    assert builder_path == str(builder_dir / "lib" / "build-labextension.js")
+    assert builder_path == str(new_builder_dir / "lib" / "build-labextension.js")