Skip to content

Add github action for testing #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
brendandahl wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add github action for testing #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d78bef8
[WIP] Enable testing on github actions.
brendandahl Jan 22, 2026
8506128
fix dev test
brendandahl Jan 22, 2026
0fe7819
can it make it in 20min?
brendandahl Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: CI

on:
push:
branches: ["main"]
pull_request:
branches: ["main"]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

jobs:
build:
name: Build
runs-on: macos-latest
strategy:
matrix:
# TODO enable safari and firefox
Copy link
Collaborator

@camillobruni camillobruni Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

side comment: I've seen that there are some resource limitations with macos so I've split the tasks in JetStream to be able to run on linux (though these are slightly slower machines).

https://github.com/WebKit/JetStream/blob/main/.github/workflows/test.yml

browser: [chrome]
steps:
- name: Install Firefox
if: ${{ matrix.browser == 'firefox' }}
run: brew install --cask firefox

- name: Checkout Branch
uses: actions/checkout@v4

- name: Setup Node
uses: actions/setup-node@v4
with:
node-version-file: "package.json"
cache: "npm"

- name: Install Node Packages
run: npm ci

- name: Download Models and Build Benchmarks
run: npm run build

- name: Run Unit Tests
run: |
echo "Running in $BROWSER"
npm run test:${{ matrix.browser }}
- name: Run end2end Tests
run: |
echo "Running in $BROWSER"
npm run test-e2e:${{ matrix.browser }}
6 changes: 3 additions & 3 deletions tests/run-end2end.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,10 @@ async function testAll() {
}
Copy link
Collaborator

@danleh danleh Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit (I unfortunately cannot add a comment to line 6, since it's too far away from the diff): Can you replace Speedometer with benchmark (both in the help text and ideally also the SpeedometerDone event)?


async function testDeveloperMode() {
const params = ["developerMode", "iterationCount=1", "warmupBeforeSync=2", "waitBeforeSync=2", "shuffleSeed=123", "suites=Perf-Dashboard"];
const params = ["developerMode", "iterationCount=1", "warmupBeforeSync=2", "waitBeforeSync=2", "shuffleSeed=123", "suites=Image-Classification-LiteRT.js-wasm"];
const metrics = await testPage(`index.html?${params.join("&")}`);
suites.forEach((suite) => {
if (suite.name === "Perf-Dashboard") {
if (suite.name === "Image-Classification-LiteRT.js-wasm") {
const metric = metrics[suite.name];
assert(metric.values.length === 1);
} else {
Expand All @@ -112,7 +112,7 @@ async function testDeveloperMode() {

async function test() {
try {
await driver.manage().setTimeouts({ script: 60000 });
await driver.manage().setTimeouts({ script: 20 * 60000 });
Copy link
Collaborator

@danleh danleh Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very small nit: Could you make 60000 a const ONE_MINUTE_IN_MS = 60000 or so, to make this a somewhat less magic number?

Also, at a high-level: 20 minutes is a really long time out, do we have a lighter workload?

await testIterations();
await testAll();
await testDeveloperMode();
Expand Down
Loading