Skip to content

feat(logseq-cli): build the OCaml/Melange CLI front-end via opam-nix#45

Merged
Bad3r merged 7 commits into
mainfrom
feat/ocaml-cli
Jun 18, 2026
Merged

feat(logseq-cli): build the OCaml/Melange CLI front-end via opam-nix#45
Bad3r merged 7 commits into
mainfrom
feat/ocaml-cli

Conversation

@Bad3r

@Bad3r Bad3r commented Jun 17, 2026

Copy link
Copy Markdown
Owner

Summary

Tracks upstream Logseq's CLI rewrite. Since logseq/logseq dbd220c95d, the CLI front-end lives under cli/, compiles OCaml 5.4 through Melange, and bundles with Vite. This PR builds that front-end inside Nix with opam-nix, keeps the spawned db-worker-node as a shadow-cljs release, and updates the desktop nightly workflow so desktop tarballs stage the same OCaml CLI runtime.

Commit stack

  1. feat(flake): add opam-nix inputs for OCaml CLI
    Pins opam-nix and opam-repository so the flake can resolve cli/logseq-cli.opam with OCaml 5.4.0.
  2. feat(logseq-cli): build OCaml CLI with opam-nix
    Reworks the Nix CLI package around dune build @bundle, the cli/ pnpm workspace, the root pnpm runtime closure, the db-worker Clojure deps, the new manifest hash, the update script, the wrapper, and the ported OAuth callback patch.
  3. ci(build-desktop): compile OCaml CLI in nightly build
    Mirrors upstream's OCaml CLI release step in the reusable desktop build workflow and verifies the staged runtime files.
  4. test(logseq-cli): match OCaml doctor output
    Updates the CLI smoke check to assert the OCaml doctor status table while keeping the db-worker spawn probe.
  5. test(desktop): require staged runtime JS assets
    Derives required ASAR /js/* runtime entries from prepare-desktop-runtime-js.mjs instead of accepting the obsolete root-entry layout.
  6. docs(logseq-cli): document OCaml CLI packaging
    Updates README.md and AGENTS.md for the OCaml/Melange CLI, cliBundlePnpmDepsHash, the desktop release step, and opam-nix IFD during evaluation.

What changed

  • Adds opam-nix and a pinned opam-repository input for the OCaml CLI closure. modules/_packages/logseq-cli/opam-deps.nix resolves OCaml 5.4.0, dune, melange*, and humanize, and links Melange's stdlib to the path that melc -where resolves under opam-nix.
  • Adds the cliBundlePnpmDepsHash manifest field plus modules/_packages/logseq-cli/cli-pnpm-deps.nix for the separate cli/ pnpm workspace (vite and transit-js). The manifest producer, validator, manifest data, and package scope were updated together.
  • Reworks modules/_packages/logseq-cli/build.nix to mirror upstream's release flow: install root pnpm dependencies offline, install the cli/ pnpm store offline, run dune build @bundle, stage static/logseq-cli.js, build and bundle the shadow-cljs db-worker-node, and run prepare-cli-package.mjs.
  • Keeps runtime native dependencies explicit. The build compiles keytar.node, prunes build-only node-gyp intermediates, preserves the Linux libsecret RPATH, and puts sqlite3 on PATH for the cli/ pnpm store extraction.
  • Wraps the CLI so it works from any current directory by setting LOGSEQ_DB_WORKER_NODE_SCRIPT to the packaged worker and prepending the bundled Node.js to PATH before executing dist/logseq.js.
  • Ports the OAuth callback patch to the OCaml platform implementation. The patch keeps the registered localhost redirect URI, binds explicit 127.0.0.1 and ::1 loopback listeners, and flushes the HTTP callback response before settling the login task.
  • Updates .github/workflows/build-desktop.yml to install the OCaml and cli/ pnpm dependencies, run opam exec -- pnpm cli:release, and verify the staged desktop runtime files (static/js/logseq-cli.js, static/js/db-worker-node.js, and the bundled Logseq CLI skill).
  • Tightens checks around the new runtime layout. logseq-runtime-assets derives required ASAR /js/* entries from prepare-desktop-runtime-js.mjs; the CLI help check matches the OCaml doctor's status table and boots the db-worker via a graph create/list probe.
  • Updates repository documentation for the OCaml/Melange CLI build and the additional manifest hash.

Validation

History cleanup validation on the rewritten head:

  • git diff --exit-code backup/feat-ocaml-cli-pre-cleanup-20260618T082948Z..HEAD
  • nix fmt -- --ci
  • Commit message scan for non-ASCII dash characters in origin/main..HEAD
  • Commit message line-length scan for lines over 120 characters in origin/main..HEAD

The rewritten branch preserves the same final file tree as the pre-cleanup head. Earlier local validation recorded for that same tree:

  • nix build .#logseq-cli
  • nix build .#checks.x86_64-linux.logseq-runtime-assets
  • nix build .#checks.x86_64-linux.logseq-cli-help
  • nix build .#checks.x86_64-linux.logseq-cli-graph-query
  • nix build .#checks.x86_64-linux.logseq-cli-login-callback
  • git apply --check and patch -p1 --dry-run for patches/logseq-cli-auth-bind-loopback-address-families.patch against logseqRev 1d1ca70d13368dafdd1f4181d5a6d0dc3946e577
  • actionlint .github/workflows/build-desktop.yml
  • nix-instantiate --parse, nixfmt --check, and deadnix --fail on changed Nix files, plus nix fmt

GitHub checks rerun on the force-pushed head ce0d01426ff428bdc3c93c0d57b1b2f417ec1ce8.

Review notes

  • This PR touches both flake.lock and .github/workflows/build-desktop.yml. The required status(security-review-approved) label is already applied.
  • opam-nix introduces IFD for the CLI dependency closure. CLI evaluation and builds can compile the OCaml 5.4.0 toolchain per system if those paths are not already cached.
  • opam-nix vendors additional nixpkgs pins through locked inputs (nixpkgs-llvm17, nixpkgs-python38, and nixpkgs-python39), so audit-flake-inputs remains the gate for unsupported or unexpected input drift.

@Bad3r Bad3r added type(enhancement) New capability or user-visible improvement. area(flake) Flake outputs, checks, formatter config, and lock/input maintenance. area(cli) logseq-cli package and upstream CLI build path. area(ci) GitHub Actions validation, workflow policy checks, and CI failures. area(deps) Dependency surfaces that are not covered by a specific input label. origin(upstream) Root cause or required fix is in upstream Logseq or upstream tooling. labels Jun 17, 2026
@claude

claude Bot commented Jun 17, 2026

Copy link
Copy Markdown

Claude finished @Bad3r's task in 5m 23s —— View job


Review

No issues found.

The prior review threads (gemini, codex, claude) cover the substantive concerns this diff raises:

  • sqlite/zstd plumbed through package.nix -> build.nix -> nativeBuildInputs (commit d536d5d)
  • Fail-loud handling of unresolved const refs in the runtime-assets spread branch (commit 5ba55ed)
  • logseqRev vs published assets invariants — maintainer decision documented, rationale captured

Spot checks against the working tree:

  • modules/_checks/runtime-assets.nix:100-105 raises SystemExit on a non-optional spread whose const cannot be resolved
  • modules/_packages/logseq-cli/build.nix:70-73 declares sqlite and zstd explicitly so the manual cli/ store extraction is self-sufficient
  • modules/_packages/logseq-cli/wrapper.nix prepends ${logseqNodejs}/bin while preserving the caller's $PATH, and pins LOGSEQ_DB_WORKER_NODE_SCRIPT only when unset
  • patches/logseq-cli-auth-bind-loopback-address-families.patch accounting (pending/bound) tracks symmetrically across the EAFNOSUPPORT/EADDRNOTAVAIL paths, listen-success paths, and the synchronous-throw path
  • scripts/update-nightly.sh writes the new cliBundlePnpmDepsHash placeholder and resolves it via the FOD-failure parser; matches lib/loadManifest.nix

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request migrates the Logseq CLI build from ClojureScript to OCaml compiled via Melange and bundled with Vite. It integrates opam-nix to resolve OCaml dependencies, introduces a separate pnpm workspace for the CLI bundle, and updates the loopback address binding patch to target the new OCaml implementation. The review feedback recommends explicitly adding and forwarding sqlite and zstd as build inputs to ensure that sqlite3 and tar --zstd are available in the sandboxed build environment.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread modules/_packages/logseq-cli/package.nix
Comment thread modules/_packages/logseq-cli/package.nix
Comment thread modules/_packages/logseq-cli/build.nix
Comment thread modules/_packages/logseq-cli/build.nix
@Bad3r Bad3r added the status(security-review-approved) Required review gate for workflow or flake.lock changes. label Jun 17, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ead90d34e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread data/logseq-nightly.json
Comment thread modules/_packages/logseq-cli/build.nix
Comment thread data/logseq-nightly.json
Comment thread modules/_packages/logseq-cli/build.nix
Comment thread data/logseq-nightly.json
@Bad3r

Bad3r commented Jun 17, 2026

Copy link
Copy Markdown
Owner Author

/gemini review

@Bad3r

Bad3r commented Jun 17, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request migrates the Logseq CLI build from ClojureScript (shadow-cljs) to OCaml compiled via Melange and bundled with Vite, aligning with upstream changes. It introduces opam-nix to resolve the OCaml/Melange dependency closure, adds a new cliBundlePnpmDepsHash field to the manifest to handle the separate cli/ pnpm workspace, and updates the build process, wrapper script, and checks accordingly. Additionally, the loopback binding patch is rewritten to target the new OCaml implementation in cli_platform.ml. There are no review comments, so no further feedback is provided.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread data/logseq-nightly.json
Comment thread modules/_packages/logseq-cli/build.nix

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 919f1bae87

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread data/logseq-nightly.json
@Bad3r Bad3r self-assigned this Jun 17, 2026
@Bad3r

Bad3r commented Jun 18, 2026

Copy link
Copy Markdown
Owner Author

Review feedback resolved

Implemented

  • sqlite/zstd build inputs (gemini + claude): plumbed sqlite and zstd package.nix -> build.nix and added to nativeBuildInputs (d536d5d). The cli/ workspace store is unpacked in buildPhase with bare tar --zstd and sqlite3. Verified against the nixpkgs hook with a standalone probe (nativeBuildInputs = [ pnpmConfigHook ]): the hook already propagates zstd onto PATH but not the sqlite3 CLI, so sqlite3 (build.nix:175) was the real missing binary. Both are declared explicitly so the manual extraction is self-sufficient rather than relying on the hook's propagation.

Not changed (rationale)

  • logseqRev bump (codex + claude): keeping logseqRev = 1d1ca70d rather than adding a cliRev field. The CLI source must lead the published desktop assets (80aeb07d has no OCaml cli/); nightlies always build desktop + CLI from the same HEAD, so a cliRev field would never diverge from logseqRev outside a manual transition like this one; and the icon / cliVersion / release-notes effects are cosmetic or one-time and self-heal at the next nightly that publishes assets at/after this rev. Full detail in the inline threads.

Validation: nix-instantiate --parse, nixfmt --check, deadnix, statix, and the pre-commit hook run all pass on the changed modules. The full nix build .#logseq-cli is left to CI (it triggers the opam-nix IFD).

Comment thread modules/_checks/runtime-assets.nix
Bad3r added 6 commits June 18, 2026 11:30
logseq/logseq dbd220c95d moved the CLI front-end to OCaml/Melange under cli/.
The melange packages and the humanize git pin are not supplied by the existing
nixpkgs input, so the flake now pins opam-nix and opam-repository for the CLI
package to resolve cli/logseq-cli.opam against OCaml 5.4.0.

The lockfile records opam-nix's own nixpkgs pins, so this change stays behind the
flake.lock sensitive-change gate.
logseq/logseq dbd220c95d replaced the shadow-cljs CLI front-end with cli/, dune
build @Bundle, Melange, and a Vite bundle. The db-worker-node process remains a
shadow-cljs release, so build.nix now combines the opam-nix OCaml closure, the
root pnpm store, the cli/ pnpm store, and the Clojure dependency FOD.

Add cliBundlePnpmDepsHash to the manifest validator, package scope, nightly
manifest producer, and generated manifest data. The wrapper also pins
LOGSEQ_DB_WORKER_NODE_SCRIPT and Node.js on PATH so doctor and graph commands can
spawn the packaged worker from any current directory.

The loopback OAuth patch moves to cli/lib/platform/node/cli_platform.ml because
the pinned revision no longer compiles src/main/logseq/cli/auth.cljs. The port
keeps both loopback address families and flushes the callback response before the
login task settles.

Validation from the pre-cleanup tree: nix build .#logseq-cli and git apply
--check for patches/logseq-cli-auth-bind-loopback-address-families.patch against
logseqRev 1d1ca70d13368dafdd1f4181d5a6d0dc3946e577.
The desktop tarballs ship static/js/logseq-cli.js, and logseq/logseq dbd220c95d
now produces that file with opam exec -- pnpm cli:release instead of the old
shadow-cljs CLI target. Mirror upstream build-desktop-release.yml by installing
OCaml 5.4.0, fetching cli/ pnpm dependencies, and running the CLI release step
between db-worker-node:bundle and webpack-app-build.

Assert static/js/logseq-cli.js and the staged Logseq CLI skill after
desktop:prepare-runtime-js so a dropped runtime staging step fails before the
packaging jobs publish artifacts.

Validation from the pre-cleanup tree: actionlint .github/workflows/build-desktop.yml.
The OCaml CLI's doctor command exits 0 with a per-check status table and does
not print the old Doctor: ok summary. Keep the smoke check strict by requiring
the db-worker-script row, then leave the graph create and list probe to exercise
the packaged db-worker-node.js process.

Validation from the pre-cleanup tree: nix build .#checks.x86_64-linux.logseq-cli-help.
The pinned Logseq runtime staging script no longer provides root-level runtime
entries to compare against, so the old check failed before it could validate the
ASAR contents. Derive each non-optional copyPairs destination under static/js/
and require the packaged ASAR to ship it as /js/<name>, which is the layout the
desktop app loads at startup.

Validation from the pre-cleanup tree: nix build .#checks.x86_64-linux.logseq-runtime-assets.
The repo documentation still described logseq-cli as a shadow-cljs CLI build with
a single pnpm dependency hash. Since logseq/logseq dbd220c95d the CLI front-end
is OCaml/Melange, while db-worker-node remains the shadow-cljs sidecar.

Document the opam-nix closure, the separate cli/ pnpm bundle hash, the updated
nightly build sequence, and the import-from-derivation effect that means
--no-build and --offline evaluation still need the opam closure built or cached.

Validation from the pre-cleanup tree: nix fmt.
Comment thread modules/_checks/runtime-assets.nix
Unresolved constants in non-optional repoRoot spread copyPairs made the ASAR drift guard skip required static/js entries. The parser now exits with the identifier and asks for a parser update instead of shrinking required-runtime-names.

Validation:

nixfmt modules/_checks/runtime-assets.nix

nix-instantiate --parse modules/_checks/runtime-assets.nix >/dev/null

nix build .#checks.x86_64-linux.logseq-runtime-assets
@Bad3r

Bad3r commented Jun 18, 2026

Copy link
Copy Markdown
Owner Author

Review feedback resolved

Accepted and implemented:

  • The two duplicate modules/_checks/runtime-assets.nix findings were valid and in scope. Commit 5ba55ed now fails loudly when a non-optional copyPairs repoRoot spread references a string const that the parser cannot resolve.
  • The error names the unresolved identifier and tells maintainers to update the runtime-assets parser instead of silently shrinking required_runtime_names.

Rejected:

  • None. No remaining fetched feedback was rejected as invalid or out of scope in this pass.

Reliability impact:

  • The ASAR drift guard now preserves fail-closed behavior when upstream changes scripts/prepare-desktop-runtime-js.mjs. An unsupported upstream declaration shape becomes an actionable check failure instead of a false green runtime-assets check.

Validation:

  • nixfmt modules/_checks/runtime-assets.nix
  • nix-instantiate --parse modules/_checks/runtime-assets.nix >/dev/null
  • nix build .#checks.x86_64-linux.logseq-runtime-assets

@Bad3r Bad3r merged commit 0e251a7 into main Jun 18, 2026
25 of 28 checks passed
@Bad3r Bad3r deleted the feat/ocaml-cli branch June 18, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area(ci) GitHub Actions validation, workflow policy checks, and CI failures. area(cli) logseq-cli package and upstream CLI build path. area(deps) Dependency surfaces that are not covered by a specific input label. area(flake) Flake outputs, checks, formatter config, and lock/input maintenance. origin(upstream) Root cause or required fix is in upstream Logseq or upstream tooling. status(security-review-approved) Required review gate for workflow or flake.lock changes. type(enhancement) New capability or user-visible improvement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant