Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions base/DESIGN.base.md

Large diffs are not rendered by default.

9 changes: 5 additions & 4 deletions docs/DESIGN.ja.md

Large diffs are not rendered by default.

9 changes: 5 additions & 4 deletions docs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -630,13 +630,13 @@ export type { CreateTOptions } from '@kiritan/runtime';

| Kind | Status in v1 |
| --- | --- |
| Translation storage strategy | `sidecar` / `inline` / `catalog` are built in, but hardcoded directly into the pipeline rather than dispatched through the declared `TranslationStore` interface. `plugins.stores` isn't wired yet — setting it currently has no effect. |
| Translation storage strategy | `sidecar` / `inline` / `catalog` are built in and hardcoded, but a source using any other `strategy` string is now genuinely dispatched through `plugins.stores[strategy]` if one is registered there (chapter 4, `TranslationStore`) — `build`/`check`/`translate`/`extract` all call into it. A `strategy` matching neither a built-in nor a registered store throws a clear error, in all four commands. |
| Runtime resource placement strategy | `colocated` / `split` / `centralized` / `embedded` (chapter 9.1) are genuinely dispatched via `ResourceSourceConfig.strategy`. A custom string strategy is accepted by the type but not handled by anything yet. |
| Translate middleware | Genuinely extensible today: no built-in providers, wired entirely through the `translate.middlewares` array (users implement freely; examples are provided in `docs/`). |
| File renderer | Only `markdown` is implemented; every source is processed as Markdown regardless of extension. `.txt`/`.mdx`, the declared `Renderer` interface, and `plugins.renderers` aren't wired yet — setting `plugins.renderers` currently has no effect. |
| File renderer | Only `markdown` is implemented; every source is processed as Markdown regardless of extension. `.txt`/`.mdx`, the declared `Renderer` interface, and `plugins.renderers` aren't wired yet — setting `plugins.renderers` currently has no effect (see chapter 13 for why this is a separate, still-unscoped piece of work from `plugins.stores` above). |
| Language-switcher rendering | Genuinely extensible today via `SwitcherConfig.render` (chapter 6.1). |

`TranslationStore`, `Renderer`, and `plugins.{stores,renderers}` are declared today as the shape a future pluggable version will use, but nothing in v1 reads or calls them yet (tracked in chapter 13).
`plugins.stores` is wired as of chapter 13; `Renderer`/`plugins.renderers` are still declared only as the shape a future pluggable version will use, with nothing in v1 reading or calling them yet.

## 12. Open questions (to be settled as implementation proceeds)

Expand All @@ -650,5 +650,6 @@ None at this time. Anything that comes up during implementation will be appended
- **A per-command `--locale` flag**: done, for `build`/`check`/`translate`/`extract` — each accepts `--locale <locale>` to restrict a run to one locale instead of every locale in `locales.list`, via a shared `resolveTargetLocales` helper that also rejects a locale not in `locales.list` with a clear error. `build`'s switcher links still cover every configured locale regardless of `--locale`, since the other locales' files already exist on disk and the switcher isn't only describing the current run. `kiritan typegen` doesn't support it — it always aggregates every locale into one runtime module, so there's no meaningful way to restrict it to one.
- **VS Code extension**: [`extensions/vscode`](../extensions/vscode) — in progress. Syntax highlighting for `:::kiritan{...}`/`::kiritan{...}` blocks (a declarative TextMate grammar injected into Markdown, no compiled extension code) is done. Published as `otoneko1102.kiritan` (a VS Code extension identifier can't contain a `/`, ruling out `@kiritan/vscode` as originally envisioned; its `package.json` `"name"` is the plain `"kiritan"`). Since that collides with the CLI package's own npm name, it lives under `extensions/` rather than `packages/` and so is never picked up by the root `"workspaces": ["packages/*"]` glob — its build/test tooling lives in the root `package.json`'s `devDependencies` instead (chapter 2.1). Highlighting for `%{name}` (and its `\%{name}` escape), folding for `:::kiritan` blocks (colon-count matched via a small stack, so nesting other directives around a kiritan block doesn't confuse it), jump-to-definition from `:::kiritan{#<id>}` to its entry in the sibling `<base>.<locale>.catalog.json`, undefined-`%{name}`-variable warnings, and inline `missing`/`stale`/`machine` indicators are all done — the last two settled the "bigger design question" the others were left waiting on: rather than requiring `kiritan`'s internals in-process (real crash/version-skew risk from a broken or mismatched project install), `diagnostics.cjs` spawns the workspace's own locally-installed `kiritan check --json` as a child process, the same approach ESLint/Prettier's editor integrations use for their own local installs. `npx --no-install` means a project that doesn't depend on `kiritan` at all is silently skipped rather than triggering a surprise network install. `kiritan check` gained a `--json` flag and `CheckIssue.id` (the catalog segment id, when there is one) specifically to support this. `%{name}` uses inside a fenced code block or inline code span are never flagged, matching `interpolation.skipCodeBlocks`'s own default — caught by testing this against the extension's own README, which shows the syntax as a literal example. Only the default `%{`/`}` delimiters are supported for the undefined-variable check; a project with custom `interpolation.delimiters` still gets everything else. It also registers `*.kiritanconfig` as its own language, delegating highlighting to `source.js` via a TextMate `include` and reusing JavaScript's bracket/comment/indent rules via its own `language-configuration.json` — this is why config files are named `*.kiritanconfig` rather than `.kiritan.mjs` (chapter 3.1): a file icon theme's own rules always beat a language's fallback icon, so keeping a real `.mjs` extension would have permanently shown a generic JavaScript icon in themes like vscode-icons. No icon theme has a rule for a filename it's never heard of, so registering the language here is enough to get a Kiritan-branded icon everywhere, with no per-theme configuration needed. Real code completion needed more than a grammar, though — the built-in TypeScript/JavaScript language service only activates for the actual `javascript`/`typescript` language ids, and giving `*.kiritanconfig` the real `javascript` id would let vscode-icons' own language-based icon rule override the custom one. `extension.cjs` bridges this: it mirrors a `*.kiritanconfig` document's content into an in-memory `javascript` document (a `TextDocumentContentProvider`-backed virtual document, not an untitled one — an untitled document is a real editable buffer that VS Code counts as unsaved, which an earlier version of this bridge did by mistake) and forwards completion requests to VS Code's own built-in provider for it, so the icon and full IntelliSense both hold at once.
- **Vim/Neovim plugin**: [`extensions/vim`](../extensions/vim) — highlighting done, everything past that still open. Ended up as plain Vimscript rather than the shared tree-sitter grammar or LSP originally envisioned here, since that's a real upfront investment and the immediate need (matching the VS Code extension's own v1) didn't call for it yet: `after/syntax/markdown.vim` highlights `:::kiritan{...}`/`::kiritan{...}` directives and `%{name}` interpolation by layering onto the built-in `markdown` syntax, the Vimscript equivalent of the VS Code extension's TextMate grammar injection, and `ftdetect/kiritanconfig.vim` assigns the real `javascript` filetype to `*.kiritanconfig` files directly — no need for VS Code's `kiritanconfig`-language workaround, since Vim/Neovim's icon ecosystem doesn't force the same vscode-icons-language-rule tradeoff (chapter 3.1). Verified against a real headless `vim -u NONE` process (`synID()`/`synIDattr()` at specific positions), not just asserted against the `.vim` source, the same rigor the VS Code extension's grammar tests use — this actually caught a real gotcha along the way: Vim's `:syntax on` autocommand only ever runs `runtime! syntax/<ft>.vim` against each `'runtimepath'` entry's own `syntax/` subdirectory, never a nested `after/syntax/`, so a plugin's `after/` subdirectory has to be added to `'runtimepath'` as its *own* separate entry (which plugin managers do automatically) for `after/syntax/markdown.vim` to load at all. Folding (`after/ftplugin/markdown.vim` sets `'foldexpr'` to `autoload/kiritan.vim`'s `kiritan#FoldExpr`, precomputing every line's fold level with the same colon-count-stack algorithm as `folding-core.cjs`, cached per buffer against `b:changedtick`) and jump-to-catalog-entry (`:KiritanJumpToCatalog`, bound to nothing by default — see the plugin's README for wiring up `<Plug>(kiritan-jump-to-catalog)`; populates the quickfix list when more than one locale's catalog matches) are done too, verified the same real-`vim`-process way. Undefined-`%{name}` detection and inline `missing`/`stale`/`machine` indicators are done too, but Neovim-only (`nvim-0.10+`) — `lua/kiritan/diagnostics.lua`, auto-loaded by `plugin/kiritan.lua`, ports the VS Code extension's `diagnostics.cjs` approach exactly: shells out to the workspace's own locally-installed `kiritan check --json` via `npx --no-install` and sets real `vim.diagnostic` entries from the result. Plain Vim has nothing resembling `vim.diagnostic`/`vim.system` to port this to — and needs no explicit guard against loading it anyway, since Vim's own runtime loader only globs `plugin/*.vim`, never `.lua`, so `plugin/kiritan.lua` is simply never read under Vim in the first place (confirmed directly: adding it left Vim's own `:scriptnames` completely unaware of its existence, no error, no side effect). Verified with a real headless `nvim --clean` process the same way as the Vimscript side, which caught a real bug before it shipped: `kiritan check` exits 1 whenever any issue matches `check.failOn` — exactly the case with something worth diagnosing — but the check's callback was discarding the result whenever the exit code was non-zero, so it would have silently reported nothing for the one scenario that mattered. The same bug, for the same reason (Node's `exec` rejects on any non-zero exit but the rejection still carries `.stdout`), was hiding in the VS Code extension's own `diagnostics.cjs` and got fixed alongside this. Releases are lighter than the other components too, since there's nothing to build: `release-vim.yml` just runs the test suite and tags the commit as `kiritan-vim@<version>` with a `--generate-notes` GitHub Release (no attached asset). There's no `package.json` here to hold a current version to bump from, so it finds the latest existing `kiritan-vim@*` tag and reuses `npm version`'s own semver logic against a throwaway `package.json` — the same `patch`/`minor`/`major`/`prerelease` keyword or explicit-version `version` input as `release.yml`.
- **Wiring `plugins.stores`/`plugins.renderers`**: making the declared `TranslationStore`/`Renderer` interfaces (chapter 11) actually pluggable, instead of `sidecar`/`inline`/`catalog` and the Markdown renderer being hardcoded into the pipeline as they are in v1.
- **Wiring `plugins.stores`**: done. `build`/`check`/`translate`/`extract` all dispatch a source whose `strategy` matches a `plugins.stores` key to that `TranslationStore`'s `read`/`write`/`status` (chapter 4, chapter 11) instead of throwing — the two `TranslatedContent` shapes it declares turned out to generalize the built-ins cleanly: a `"full-text"` result is treated sidecar-style (parsed fresh as the whole document), a `"segments"` result (or nothing stored yet) catalog-style (fed into `renderForLocale`'s existing per-id fallback, so no new fallback logic was needed). Staleness is left entirely to the store's own `status()` — `check`/`translate` trust it outright rather than trying to recompute it the way the built-in `sidecar`/`catalog` formats do for themselves, since a plugin store is a black box kiritan doesn't own the internals of. `check` only calls `status()`, not `read()`, so there's no generic "machine-translated, needs review" issue for a plugin store the way `catalog`'s own `machine` flag produces one — that stays a `catalog`-specific concept for now. `translate`/`extract` both no-op for a store with no `write` (a read-only store, e.g. one backed by an external TMS kiritan can only read from). A `strategy` matching neither a built-in nor a registered store now throws in all four commands, not just `build` as before — a deliberate tightening, since a custom `strategy` typo (e.g. `"catalogg"`) was always a mistake with no legitimate use, and there's now an actual registration mechanism to check it against.
- **Wiring `plugins.renderers`**: still not done, and deliberately scoped separately from `plugins.stores` above — the declared `Renderer` interface (`parse(sourceText): { raw: string }`, `reassemble(doc, translated: string)`) has no way to represent the mdast tree `directive/render.ts`'s `renderForLocale`/`collectCatalogSegments`/etc. actually operate on end-to-end, so it can't be wired against the real pipeline as declared. A working design needs an actual second renderer (see the next item) to validate the interface's real shape against, not just the bare `strategies: StoreStrategy[]` field it already has.
- **`text`/`mdx` renderers and front-matter protection**: v1 only implements the `markdown` renderer; `.txt`/`.mdx` support and `translateFrontmatter` (chapter 6) are design-stage only.
94 changes: 93 additions & 1 deletion packages/kiritan/src/pipeline/build.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,32 @@ import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import type { KiritanConfig } from "../config/types.js";
import type {
KiritanConfig,
TranslatedContent,
TranslationStore,
} from "../config/types.js";
import { build } from "./build.js";

/** A minimal in-memory `TranslationStore` fake, keyed by locale. */
function createMemoryStore(
initial: Record<string, TranslatedContent> = {}
): TranslationStore {
const data = new Map(Object.entries(initial));
return {
id: "memory",
async read(_ctx, locale) {
return data.get(locale) ?? null;
},
async write(_ctx, locale, content) {
data.set(locale, content);
},
async status(_ctx, locale) {
return data.has(locale) ? "complete" : "missing";
},
};
}

let cwd: string;

beforeEach(async () => {
Expand Down Expand Up @@ -223,3 +246,72 @@ describe("build (catalog strategy)", () => {
expect(ja).toContain("Original text.");
});
});

describe("build (plugins.stores)", () => {
it("renders a full-text store's translation as its own document", async () => {
await writeFile(join(cwd, "README.base.md"), "Original text.", "utf8");
const store = createMemoryStore({
ja: { kind: "full-text", text: "翻訳済みテキスト。" },
});
const config = baseConfig({
sources: [{ glob: "README.base.md", strategy: "memory" }],
plugins: { stores: { memory: store } },
});

const result = await build(config, { cwd });
expect(result.written.sort()).toEqual(["README.ja.md", "README.md"]);
const ja = await readFile(join(cwd, "README.ja.md"), "utf8");
expect(ja).toContain("翻訳済みテキスト。");
expect(ja).not.toContain("Original text.");
});

it("resolves each #id from a segments store, falling back to the source text when missing", async () => {
await writeFile(
join(cwd, "README.base.md"),
[":::kiritan{#intro}", "Original text.", ":::"].join("\n"),
"utf8"
);
const store = createMemoryStore({
ja: {
kind: "segments",
segments: { intro: { text: "翻訳済みテキスト。" } },
},
});
const config = baseConfig({
sources: [{ glob: "README.base.md", strategy: "memory" }],
plugins: { stores: { memory: store } },
});

await build(config, { cwd });
const ja = await readFile(join(cwd, "README.ja.md"), "utf8");
expect(ja).toContain("翻訳済みテキスト。");
expect(ja).not.toContain("Original text.");
});

it("falls back to the source text with a marker when nothing is stored yet", async () => {
await writeFile(
join(cwd, "README.base.md"),
[":::kiritan{#intro}", "Original text.", ":::"].join("\n"),
"utf8"
);
const config = baseConfig({
sources: [{ glob: "README.base.md", strategy: "memory" }],
plugins: { stores: { memory: createMemoryStore() } },
});

await build(config, { cwd });
const ja = await readFile(join(cwd, "README.ja.md"), "utf8");
expect(ja).toContain("kiritan:untranslated (source: en)");
expect(ja).toContain("Original text.");
});

it("throws for a strategy with no matching plugins.stores entry", async () => {
await writeFile(join(cwd, "README.base.md"), "hello", "utf8");
const config = baseConfig({
sources: [{ glob: "README.base.md", strategy: "unregistered" }],
});
await expect(build(config, { cwd })).rejects.toThrow(
/the "unregistered" strategy isn't implemented yet/
);
});
});
64 changes: 64 additions & 0 deletions packages/kiritan/src/pipeline/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import type {
BuildContext,
KiritanConfig,
SourceConfig,
StoreContext,
SwitcherConfig,
} from "../config/types.js";
import { resolveTargetLocales } from "../config/locale.js";
Expand Down Expand Up @@ -190,6 +191,69 @@ export async function build(
continue;
}

const store = config.plugins?.stores?.[file.source.strategy];
if (store) {
const baseTree = ensureSwitcherMarker(
parseMarkdown(sourceText),
switcherConfig
);
const ctx: StoreContext = { source: file.source, filePath: file.path };

for (const locale of targetLocales) {
const outPath = outputPathFor(locale);

if (locale === config.locales.default) {
const rendered = renderForLocale(baseTree, {
targetLocale: locale,
defaultLocale: config.locales.default,
renderSwitcher: renderSwitcherFor(locale, outPath),
});
await writeOutput(
cwd,
outPath,
finalizeTree(rendered, config, locale)
);
written.push(outPath);
continue;
}

const content = await store.read(ctx, locale);

// A "full-text" result replaces the whole document, the same way a sidecar file's own content does — it's parsed fresh rather than layered onto the base tree.
if (content?.kind === "full-text") {
const tree = ensureSwitcherMarker(
parseMarkdown(content.text),
switcherConfig
);
const rendered = renderForLocale(tree, {
targetLocale: locale,
defaultLocale: config.locales.default,
renderSwitcher: renderSwitcherFor(locale, outPath),
});
await writeOutput(
cwd,
outPath,
finalizeTree(rendered, config, locale)
);
written.push(outPath);
continue;
}

// A "segments" result (or nothing stored yet) is treated catalog-style — renderForLocale's own per-id fallback handles a missing id or a null/undefined content.
const segments =
content?.kind === "segments" ? content.segments : undefined;
const rendered = renderForLocale(baseTree, {
targetLocale: locale,
defaultLocale: config.locales.default,
resolveCatalogText: segments ? (id) => segments[id]?.text : undefined,
renderSwitcher: renderSwitcherFor(locale, outPath),
});
await writeOutput(cwd, outPath, finalizeTree(rendered, config, locale));
written.push(outPath);
}
continue;
}

throw new Error(
`kiritan: the "${file.source.strategy}" strategy isn't implemented yet`
);
Expand Down
Loading
Loading