Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,31 +12,35 @@ jobs:
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
# Installs the pnpm version pinned by package.json's `packageManager` field; must run
# before setup-node so its `cache: pnpm` can find the pnpm store.
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 24
cache: npm
- run: npm ci
- run: npm run build
- run: npm run lint
- run: npm run format:check
- run: npm test
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm run build
- run: pnpm run lint
- run: pnpm run format:check
- run: pnpm test

e2e:
name: Actor dev-loop e2e (apify-cli + Docker)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 24
cache: npm
- run: npm ci
cache: pnpm
- run: pnpm install --frozen-lockfile
# The suite manages Docker itself against the runner's daemon: it pre-pulls the
# Actor base images, builds the runtime image, starts the runtime container with
# the host Docker socket, and drives it with stock apify-cli via npx.
- run: npm run test:e2e
- run: pnpm run test:e2e
# The e2e's runtime container is normally removed by the suite's afterAll; on
# failure it is left running, so its server-side view of any failed request
# (log-stream lifecycle included) is captured here for diagnosis.
Expand Down
3 changes: 3 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# pnpm's generated lockfile is pnpm's own output format, not ours to reformat - and the next
# `pnpm install` would rewrite it in that format anyway.
pnpm-lock.yaml
23 changes: 14 additions & 9 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,25 +1,30 @@
# actor-runtime: a minimal, self-contained local Apify platform.
#
# glibc is required, not optional: @crawlee/fs-storage loads a native Rust addon
# (@crawlee/fs-storage-native) with no musl build, so this image cannot be Alpine-based
# (see requirements/system.md and file-system-storage.ts:19-27 in the crawlee v4 source).

FROM node:24-bookworm-slim AS builder

WORKDIR /usr/src/app

COPY package.json package-lock.json ./
RUN npm ci
ENV COREPACK_ENABLE_DOWNLOAD_PROMPT=0
RUN corepack enable pnpm

COPY package.json pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile

COPY tsconfig.json ./
COPY src ./src
RUN npm run build
RUN pnpm run build

FROM node:24-bookworm-slim

WORKDIR /usr/src/app

COPY package.json package-lock.json ./
RUN npm ci --omit=dev && npm cache clean --force
ENV COREPACK_ENABLE_DOWNLOAD_PROMPT=0
RUN corepack enable pnpm

COPY package.json pnpm-lock.yaml ./
# The store prune plays the role `npm cache clean` played before: production node_modules keeps
# hard links into the store, so pruning drops only the unreferenced (dev) packages' disk copies.
RUN pnpm install --prod --frozen-lockfile && pnpm store prune

COPY --from=builder /usr/src/app/dist ./dist

Expand Down
26 changes: 16 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,14 @@ endpoint/console details: `requirements/api.md`'s `/actor-runtime/*` section and
## Development

```bash
npm install
npm run build # tsc
npm test # unit + integration (no Docker needed)
npm run test:e2e # full CLI-driven dev loop against a built image (requires Docker)
npm run dev # run the server directly against ./data with tsx
pnpm install
pnpm run build # tsc
pnpm test # unit + integration (no Docker needed)
pnpm run test:e2e # full CLI-driven dev loop against a built image (requires Docker)
pnpm run dev # run the server directly against ./data with tsx
```

`npm run dev` sets `ACTOR_RUNTIME_DATA_DIR=./data` inline in the script (`DEFAULT_DATA_DIR` otherwise
`pnpm run dev` sets `ACTOR_RUNTIME_DATA_DIR=./data` inline in the script (`DEFAULT_DATA_DIR` otherwise
falls back to the container path `/data` - see `src/config.ts`); this only works as written on a
POSIX shell (Linux/macOS). On Windows, set the env var separately before running `tsx src/index.ts`
(e.g. in PowerShell: `$env:ACTOR_RUNTIME_DATA_DIR="./data"; tsx src/index.ts`), or use a cross-platform
Expand All @@ -96,13 +96,19 @@ resolves to (both must move in lockstep - `@crawlee/fs-storage` pins its own nat
`@crawlee/fs-storage-native`). To bump:

```bash
npm view @crawlee/core dist-tags.v4
npm view @crawlee/fs-storage dist-tags.v4 # should match
pnpm view @crawlee/core dist-tags.v4
pnpm view @crawlee/fs-storage dist-tags.v4 # should match
# update both versions in package.json, then:
npm install
npm run build && npm test
pnpm install
pnpm run build && pnpm test
```

While bumping, check whether the `pnpm.overrides` pin on `@crawlee/fs-storage-native` in
`package.json` is still needed: it forces the first release with linux-arm64 bindings
(`0.1.5-beta.19`, API-identical to the `0.1.5-beta.18` that released `@crawlee/fs-storage`
versions still depend on). Once the bumped `@crawlee/fs-storage` depends on `>= 0.1.5-beta.19`
on its own, delete the override.

## Apify Proxy

Set `APIFY_PROXY_PASSWORD` in the runtime container's own environment (e.g. `docker run -e
Expand Down
Loading
Loading