Refactor package build lifecycle and improve resource management#71
Merged
Refactor package build lifecycle and improve resource management#71
Conversation
Replaced prepack with prepare in publishable package manifests so build runs in git-based installs and standard packaging flows. This keeps direct GitHub package installs working while avoiding reliance on prepack-only behavior.
Added a root TypeScript solution config for package references and a root typecheck script that runs tsc build mode with noEmit. Also updated verify to run typecheck first so type errors in package tests are caught consistently across the monorepo.
This reverts commit 37382b7.
Replace redundant double-step TypeScript builds ("tsc --build --clean && tsc --build") with a single "tsc --build" invocation across packages (common, nodejs, react-native, react-web, react, svelte, vue, web, apps/relay). Update svelte build to remove the extra tsc pass. Add a root script "clean:ts" (tsc --build --clean tsconfig.typecheck.json) and run it before the monorepo build by prepending it to the root "build" script.
Remove runtime dependency on disposablestack and implement an owned DisposableStack/AsyncDisposableStack polyfill in packages/common/src/Polyfills.ts. The polyfill installs Symbol.dispose / Symbol.asyncDispose and SuppressedError, includes a WebKit completion-bug fix (es-shims/DisposableStack#9), and exposes behavior-parity with Node where needed. Add comprehensive conformance tests in packages/common/test/Polyfills.test.ts (ports upstream es-shims/test262 checks and adds Evolu-specific regressions). Also update package.json to drop the disposablestack dependency and adjust browser test setup.
Introduce keyed concurrency registries and refactor instance lifecycle handling: add createSemaphoreByKey/createMutexByKey and extend Semaphore with withPermits and snapshot for weighted/FIFO semantics; move TaskInstances into Task.ts and implement a Task-based instances registry using mutexes per key. Refactor Instances disposal to use DisposableStack (LIFO) and adapt Resources, Relay and Shared to use the new keyed mutex API. Improve Task docs and yield/timeout fallbacks, and update tests to cover semaphore snapshots, weighted permits, scheduler fallbacks, and the revised disposal error representation. Commit before further refactor. Fixes memory leak in Relay (Instances with Mutexes was a stupid design).
Refactor the evolu ESLint rule to recursively detect CallExpression and NewExpression nodes inside exported const initializers (not just top-level calls). The rule now traverses node children via sourceCode.visitorKeys, skips function boundaries and IIFEs, treats arrow/function expressions as non-pure callee cases, and inserts /*#__PURE__*/ annotations where missing. Updated rule messages/descriptions accordingly and added comprehensive tests (scripts/eslint-plugin-evolu.test.mts) covering top-level, nested calls, new expressions, function bodies, and IIFEs. Also disabled jsdoc/check-alignment in eslint.config.mjs with a TODO about Copilot inserting extra spaces.
Large refactor to clarify Run/Task/Fiber concepts and rename concurrency helpers. Key changes: replace FiberState/FiberSnapshot with RunState/RunSnapshot and rename related types/events; introduce Run.create and adjust daemon semantics; replace runClosing* with runStopped* errors/abort reasons. Add AbortMask/Concurrency types earlier, and rename parallel->concurrently (plus updated callers). Remove TaskInstances/createTaskInstances and createDeferreds APIs and add MutexRef/createMutexRef + createMutex utilities (with snapshot support). Various doc and capitalization fixes to use consistent "Run"/"Task"/"Fiber" terminology and adjust microtask/abort semantics.
Replace the previous createTaskInstances approach with an explicit Map plus a per-name mutex (createMutexByKey) to avoid races when creating SharedEvolu instances. SharedEvolu instances are now stored in sharedEvolusByName and created under sharedEvolusMutexByName.withLock; deletion on dispose is also performed under the same lock. Refactor createSharedEvolu to derive a child console from run.deps, extract addPorts into a standalone function, and make message switch defaults use exhaustiveCheck. Misc: adjust imports (remove Console type, add exhaustiveCheck and createMutexByKey) and tidy message handling and lifecycle logic.
Merged the Evolu SQLite JSON helpers into Query.ts, removed the root kysely namespace export, and exposed the helpers as explicit named exports. Also fixed recursive parsing of nested Evolu-marked JSON payloads, updated tests to cover the merged helper implementation, and refreshed docs, playgrounds, examples, and the changeset for the new API.
Simplify createSqlite by replacing manual result handling with run.orThrow when creating the Sqlite driver. This removes the intermediate result variable and explicit ok check, directly assigning the driver and letting run.orThrow propagate errors.
Update documentation across Result.ts and Task.ts to clarify when to wrap thrown errors in Result (use trySync/tryAsync for recoverable, actionable errors) versus letting unrecoverable errors propagate to a global handler. Add guidance to prefer platform-specific createRun adapters (e.g. @evolu/web, @evolu/nodejs, @evolu/react-native) for global error handling. Remove the unused UnknownError import and the large example surrounding it, and drop a stray comment about Promise.try. Small cleanup focused on docs and imports — no behavioral changes to runtime logic.
Document preferred imperative style for composing sequential Tasks (using await/run) to avoid pipe/chain API duplication. Update AsyncDisposableStack.use signature to return PromiseLike<Result<T, E | AbortError>> to reflect abort semantics, and adjust the test to assert the resulting type is Result<Resource, AbortError>.
miccy
added
upstream
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (79)
📝 WalkthroughWalkthroughZavedení nových resource- a lifecycle-API (Resource, SharedResource, ResourceRef), nový strukturálně-klíčovaný Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant App as App/Client
participant ResourceRef as ResourceRef
participant AsyncStack as AsyncDisposableStack
participant Task as Task/Run
App->>ResourceRef: get()
ResourceRef->>Task: vytvoří/vrátí Task<BorrowedResource>
Task-->>ResourceRef: BorrowedResource (resource + stack)
ResourceRef-->>App: BorrowedResource
App->>ResourceRef: set(newCreate)
ResourceRef->>AsyncStack: dispose(stará resource)
AsyncStack->>Task: invoke [Symbol.asyncDispose]
Task-->>AsyncStack: dokončeno
ResourceRef-->>App: ok
App->>AsyncStack: dispose()
AsyncStack->>ResourceRef: [Symbol.asyncDispose]
ResourceRef-->>AsyncStack: cleanup
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minut Possibly related PRs
Suggested labels
Suggested reviewers
Poem
✨ Finishing Touches
🧪 Generate unit tests (beta)
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors resource management across the monorepo, moving several platform/runtime abstractions (SQLite, WebSocket, leader locks, runs) to async disposal patterns and introducing new primitives (e.g., StructuralMap, Resource). It also updates tests and CI configuration to match the new lifecycle semantics and bumps a number of dependencies.
Changes:
- Switched multiple runtime resources from
Symbol.disposetoSymbol.asyncDisposeand adoptedDisposableStack/AsyncDisposableStackpatterns. - Introduced
StructuralMap(structural-keyed map) and newResourcelifecycle primitives; removed older listener/resource utilities. - Updated Vitest project naming/config and refreshed various dependencies and example templates.
Reviewed changes
Copilot reviewed 78 out of 80 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/web/test/Sqlite.test.ts | Updates web SQLite tests for async disposal and installs common polyfills. |
| packages/web/test/Platform.test.ts | Simplifies leader lock tests; switches to async disposal. |
| packages/web/src/Task.ts | Refactors web createLeaderLock and renames runner types/APIs to Run. |
| packages/web/src/Sqlite.ts | Uses DisposableStack and improves prepared statement lifecycle/reset behavior. |
| packages/svelte/package.json | Bumps Svelte version and peer dependency floor. |
| packages/react-native/src/sqlite-drivers/createOpSqliteDriver.ts | Wraps op-sqlite driver lifetime in DisposableStack; updates export error message behavior. |
| packages/react-native/src/sqlite-drivers/createExpoSqliteDriver.ts | Wraps expo-sqlite driver lifetime in DisposableStack. |
| packages/react-native/src/Task.ts | Renames createRunner → createRun and updates docs/aliases for compatibility. |
| packages/react-native/package.json | Bumps op-sqlite and Expo versions. |
| packages/nodejs/test/Sqlite.test.ts | Updates Node.js SQLite tests to use async disposal. |
| packages/nodejs/src/local-first/Relay.ts | Refactors relay startup to use AsyncDisposableStack and updated run/sqlite lifecycle. |
| packages/nodejs/src/Task.ts | Renames createRunner → createRun and updates docs/aliases for compatibility. |
| packages/nodejs/package.json | Bumps better-sqlite3 and Node type definitions. |
| packages/common/vitest.unit.config.ts | Renames the unit test project for clearer targeting in CI. |
| packages/common/vitest.browser.config.ts | Adjusts browser-instance selection for VS Code Vitest coverage runs; renames project. |
| packages/common/test/local-first/Sync.test.ts | Adds coverage for socket disposal failures during teardown. |
| packages/common/test/local-first/Shared.test.ts | Adds TODO note about switching setup to a Run-with-deps. |
| packages/common/test/local-first/Evolu.test.ts | Updates inline snapshots due to changed deterministic outputs. |
| packages/common/test/local-first/Db.internal.test.ts | Adds assertions for stored-bytes accounting using protocol encoding size. |
| packages/common/test/_deps.ts | Clarifies sqlite driver comment and simplifies helper function body. |
| packages/common/test/WebSocket.test.ts | Adds testCreateWebSocket coverage and converts tests to async disposal. |
| packages/common/test/Type.test.ts | Avoids type/value naming clash in test by renaming local binding. |
| packages/common/test/TreeShaking.test.ts | Updates size baselines for fixtures. |
| packages/common/test/StructuralMap.test.ts | Adds test coverage for new StructuralMap. |
| packages/common/test/Store.test.ts | Adds test ensuring listener notification uses a snapshot during iteration. |
| packages/common/test/Sqlite.test.ts | Updates disposal semantics tests and adds LIKE-wildcard escaping coverage for schema filtering. |
| packages/common/test/Resources.test.ts | Removes legacy resources tests (file deleted). |
| packages/common/test/Ref.test.ts | Renames “state” wording to “value” in test names. |
| packages/common/test/Listeners.test.ts | Removes Listeners tests (file deleted). |
| packages/common/test/Function.test.ts | Removes Function utilities tests tied to removed readonly helper (file deleted). |
| packages/common/test/Buffer.test.ts | Updates assertions to use toThrow consistently. |
| packages/common/test/Assert.test.ts | Adds coverage for assertNotAborted / assertNotDisposed and improves typing assertions. |
| packages/common/src/local-first/Worker.ts | Refactors subscription teardown logic in deprecated worker init. |
| packages/common/src/local-first/Sync.ts | Moves WebSocket resources to async disposal with best-effort error logging on disposal. |
| packages/common/src/local-first/Shared.ts | Removes transport resource abstraction and adjusts shared worker lifecycle handling. |
| packages/common/src/local-first/Schema.ts | Removes readonly helper usage and makes exported sets/arrays explicitly readonly-typed. |
| packages/common/src/local-first/Relay.ts | Refactors quota/write logic and modifies relay storage table creation semantics. |
| packages/common/src/local-first/Evolu.ts | Refactors Evolu config docs, export typing, error-store handling, and disposal stack usage. |
| packages/common/src/local-first/Db.ts | Refactors DB worker init/storage logic, adds client storage helpers, and updates timestamp/quota handling. |
| packages/common/src/index.ts | Removes Listeners export; re-exports new Resource and StructuralMap modules; adjusts type exports. |
| packages/common/src/WebSocket.ts | Makes WebSocket async-disposable, improves ready-state mapping and expands testCreateWebSocket. |
| packages/common/src/Types.ts | Renames CallbackWithCleanup → CallbackWithTeardown. |
| packages/common/src/Type.ts | Expands orThrow documentation and adds JsonValue / JsonValueInput docs. |
| packages/common/src/Test.ts | Renames test runner helpers (testCreateRun) and updates deterministic test helper APIs. |
| packages/common/src/StructuralMap.ts | Adds StructuralMap implementation for structural-key registries (new file). |
| packages/common/src/Store.ts | Inlines listener management and snapshot notification; removes dependency on Listeners. |
| packages/common/src/Sqlite.ts | Makes Sqlite async-disposable, adds disposed-guarding for sync methods, and improves schema snapshot options. |
| packages/common/src/Result.ts | Refines documentation wording for Result composition and getOrThrow. |
| packages/common/src/Resources.ts | Extends resources to dispose both sync and async disposables; adjusts overload implementation. |
| packages/common/src/Resource.ts | Adds new resource lifecycle primitives (ResourceRef, SharedResource, keyed shared resources) (new file). |
| packages/common/src/Ref.ts | Clarifies immutability expectations in docs and renames state/value terminology. |
| packages/common/src/Random.ts | Refactors Arc4 snapshot typing into a named interface. |
| packages/common/src/Listeners.ts | Removes Listeners implementation (file deleted). |
| packages/common/src/Function.ts | Removes readonly helper and related type-only imports. |
| packages/common/src/Assert.ts | Adds assertNotAborted and assertNotDisposed. |
| packages/common/src/Polyfills.ts | Adds installPolyfills and owned DisposableStack polyfill logic. |
| packages/common/package.json | Adds ./polyfills export and bumps Kysely/better-sqlite3. |
| package.json | Bumps Biome and Turbo. |
| examples/vue-vite-pwa/package.json | Bumps @vitejs/plugin-vue. |
| examples/tauri/package.json | Bumps @vitejs/plugin-react. |
| examples/tanstack-start/package.json | Bumps TanStack Router and @vitejs/plugin-react. |
| examples/svelte-vite-pwa/package.json | Bumps Svelte. |
| examples/react-vite-pwa/package.json | Bumps @vitejs/plugin-react. |
| examples/react-nextjs/package.json | Bumps Next.js and Node types. |
| examples/react-expo/package.json | Bumps Expo-related dependencies. |
| examples/react-electron/package.json | Bumps @vitejs/plugin-react and electron vite plugins. |
| examples/astro/package.json | Bumps Astro + React integration and adds @astrojs/check. |
| examples/angular-vite-pwa/package.json | Bumps Angular packages. |
| biome.json | Updates Biome schema URL to match bumped Biome version. |
| apps/relay/src/startBunRelay.ts | Refactors relay startup stack usage and sqlite acquisition; simplifies shutdown defers. |
| apps/relay/src/index.ts | Switches relay startup to createRun and removes Bun-runtime branching. |
| apps/relay/package.json | Bumps Node types. |
| .gitignore | Ignores .vitest-attachments. |
| .github/workflows/ws-browser-nightly.yaml | Updates Vitest project name used in nightly browser WebSocket tests. |
| .github/workflows/ci.yaml | Updates Vitest project name used in CI smoke tests. |
| .github/copilot-instructions.md | Updates testing guidance wording. |
| .changeset/structural-map-uint8array.md | Adds changeset for StructuralMap minor release. |
Comments suppressed due to low confidence (1)
packages/common/src/local-first/Relay.ts:321
createRelayStorageTablesis no longer idempotent (create tableinstead ofcreate table if not exists). This breaks callers that run table creation on every startup (e.g.apps/relay/src/startBunRelay.tscurrently calls it unconditionally) and makes repeated initialization fail. Restoreif not exists(or otherwise ensure callers only run this on a fresh DB).
Comment on lines
34
to
77
| @@ -67,11 +73,14 @@ export const createLeaderLock = (): LeaderLock => ({ | |||
|
|
|||
| if (state === "failed") { | |||
| release.resolve(); | |||
| return run(inMemoryLeaderLock.acquire(name)); | |||
| return run(unabortable(inMemoryLeaderLock.lock(name))); | |||
| } | |||
Comment on lines
1
to
30
| import { Name, testName } from "@evolu/common"; | ||
| import { afterEach, describe, expect, test, vi } from "vitest"; | ||
| import { describe, expect, test } from "vitest"; | ||
| import { createLeaderLock, createRun } from "../src/Task.js"; | ||
|
|
||
| afterEach(() => { | ||
| vi.restoreAllMocks(); | ||
| }); | ||
|
|
||
| const withNavigator = async ( | ||
| navigator: typeof globalThis.navigator | undefined, | ||
| runTest: () => Promise<void>, | ||
| ): Promise<void> => { | ||
| const originalNavigator = globalThis.navigator; | ||
| describe("leaderLock", () => { | ||
| test("acquire waits until previous lease is disposed", async () => { | ||
| await using run = createRun(); | ||
| const leaderLock = createLeaderLock(); | ||
|
|
||
| Object.defineProperty(globalThis, "navigator", { | ||
| configurable: true, | ||
| writable: true, | ||
| value: navigator, | ||
| }); | ||
| const first = await run(leaderLock.lock(testName)); | ||
| expect(first.ok).toBe(true); | ||
| if (!first.ok) return; | ||
|
|
||
| try { | ||
| await runTest(); | ||
| } finally { | ||
| Object.defineProperty(globalThis, "navigator", { | ||
| configurable: true, | ||
| writable: true, | ||
| value: originalNavigator, | ||
| let secondSettled = false; | ||
| const second = run(leaderLock.lock(testName)); | ||
| void second.then(() => { | ||
| secondSettled = true; | ||
| }); | ||
| } | ||
| }; | ||
|
|
||
| const expectSequentialAcquireForSameName = async (): Promise<void> => { | ||
| await using run = createRun(); | ||
| const leaderLock = createLeaderLock(); | ||
|
|
||
| const first = await run(leaderLock.acquire(testName)); | ||
| expect(first.ok).toBe(true); | ||
| if (!first.ok) return; | ||
|
|
||
| let secondSettled = false; | ||
| const second = run(leaderLock.acquire(testName)); | ||
| void second.then(() => { | ||
| secondSettled = true; | ||
| }); | ||
|
|
||
| await Promise.resolve(); | ||
| expect(secondSettled).toBe(false); | ||
|
|
||
| first.value[Symbol.dispose](); | ||
| await Promise.resolve(); | ||
| expect(secondSettled).toBe(false); | ||
|
|
||
| const secondResult = await second; | ||
| expect(secondResult.ok).toBe(true); | ||
| if (!secondResult.ok) return; | ||
| await first.value[Symbol.asyncDispose](); | ||
|
|
||
| secondResult.value[Symbol.dispose](); | ||
| }; | ||
| const secondResult = await second; | ||
| expect(secondResult.ok).toBe(true); | ||
| if (!secondResult.ok) return; | ||
|
|
||
| describe("leaderLock", () => { | ||
| test("acquire waits until previous lease is disposed", async () => { | ||
| await expectSequentialAcquireForSameName(); | ||
| await secondResult.value[Symbol.asyncDispose](); | ||
| }); |
Comment on lines
+7
to
+9
| import { assert } from "./Assert.js"; | ||
| import { isPlainObject } from "./Object.js"; | ||
| import { type JsonValue, uint8ArrayToBase64Url } from "./Type.js"; |
Comment on lines
+161
to
164
| const lifetime = run.create(); | ||
| lifetime.onAbort(() => { | ||
| unsubscribe(); | ||
| return ok(); | ||
| }); |
Comment on lines
+542
to
544
| // eslint-disable-next-line @typescript-eslint/require-await | ||
| [Symbol.asyncDispose]: async () => { | ||
| if (appOwner) { | ||
| await run(transports.removeConsumer(appOwner, ownerTransports)); | ||
| } | ||
|
|
||
| clearActiveLeaderTimeout(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
Poznámky k vydání
Nové funkce
Vylepšení
Chores