Skip to content

Commit 93df581

Browse files
hzygithub-actions[bot]
hzy
authored and
github-actions[bot]
committed
[Feature] Introduce benchx_cli for benchmark
Use it with `./benchx_cli -o <output_ptrace_file> run <lynx_bundle_path>` to get a ptrace file of the Lynx Bundle. Under the hood it runs the Lynx Bundle with specific embedder api mocked so not suitable for serious e2e test, but should be sufficient for benchmark. Also, codspeed is integrate into the cli, so benchmark result can be uploaded to codspeed platform for tracking ingression.
1 parent 3d6e499 commit 93df581

34 files changed

+1206
-23
lines changed

‎.github/disabled_workflows.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Disabled Workflows Log (lynx-community/benchx_cli)
2+
3+
The following GitHub workflows were disabled to isolate the effects of the last commit (which modified `Benchx CLI Publish`).
4+
5+
| ID | Name | Previous State | Command to Enable |
6+
| --------- | ------------------------- | -------------- | ----------------------------------------------------------- |
7+
| 233260258 | ci | active | `gh workflow enable 233260258 -R lynx-community/benchx_cli` |
8+
| 233260259 | CocoaPods Token Keepalive | active | `gh workflow enable 233260259 -R lynx-community/benchx_cli` |
9+
| 233260260 | CodeQL Advanced | active | `gh workflow enable 233260260 -R lynx-community/benchx_cli` |
10+
| 233260261 | css_defines_publish | active | `gh workflow enable 233260261 -R lynx-community/benchx_cli` |
11+
| 233260262 | issue_management | active | `gh workflow enable 233260262 -R lynx-community/benchx_cli` |
12+
| 233260263 | lynx_core_publish | active | `gh workflow enable 233260263 -R lynx-community/benchx_cli` |
13+
| 233260264 | lynx_types_publish | active | `gh workflow enable 233260264 -R lynx-community/benchx_cli` |
14+
| 233260265 | publish-release | active | `gh workflow enable 233260265 -R lynx-community/benchx_cli` |
15+
| 233260266 | sdk-release-test | active | `gh workflow enable 233260266 -R lynx-community/benchx_cli` |
16+
| 233260267 | tasm_publish | active | `gh workflow enable 233260267 -R lynx-community/benchx_cli` |
17+
| 233260268 | type_config_publish | active | `gh workflow enable 233260268 -R lynx-community/benchx_cli` |
18+
| 233260270 | type_element_api_publish | active | `gh workflow enable 233260270 -R lynx-community/benchx_cli` |

‎.github/workflows/benchx_cli_publish.yml‎

Lines changed: 253 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,253 @@
1+
name: Benchx CLI Publish
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- develop
7+
push:
8+
tags:
9+
- '*'
10+
workflow_call:
11+
inputs:
12+
skip_release:
13+
description: 'Skip the release job'
14+
required: false
15+
default: false
16+
type: boolean
17+
commit_ref:
18+
description: 'Commit SHA or branch name to build'
19+
required: false
20+
type: string
21+
workflow_dispatch:
22+
23+
permissions:
24+
contents: write
25+
26+
jobs:
27+
build:
28+
name: Build ${{ matrix.os_name }} ${{ matrix.arch_name }}
29+
runs-on: ${{ matrix.os }}
30+
env:
31+
SCCACHE_GHA_ENABLED: 'true'
32+
strategy:
33+
fail-fast: false
34+
matrix:
35+
include:
36+
- os: ubuntu-latest
37+
os_name: Linux
38+
arch_name: x86_64
39+
- os: macos-latest
40+
os_name: Darwin
41+
arch_name: arm64
42+
43+
steps:
44+
- name: Checkout
45+
uses: actions/checkout@v4
46+
with:
47+
ref: ${{ inputs.commit_ref || github.ref }}
48+
49+
- name: Setup Node
50+
uses: actions/setup-node@v4
51+
with:
52+
node-version: '24'
53+
54+
- name: Setup sccache
55+
uses: mozilla-actions/sccache-action@v0.0.8
56+
57+
- name: Setup uv
58+
uses: astral-sh/setup-uv@v5
59+
60+
- name: Install Dependencies
61+
run: |
62+
# Configure sccache
63+
echo "SCCACHE_DIR=$GITHUB_WORKSPACE/.sccache" >> $GITHUB_ENV
64+
mkdir -p .sccache
65+
66+
# Show sccache stats before build
67+
sccache -z
68+
69+
uv venv
70+
uv pip install pip
71+
72+
# Ensure tools are executable
73+
chmod +x tools/hab || true
74+
75+
# Source envsetup to set paths for hab and other tools
76+
set +u
77+
source tools/envsetup.sh
78+
79+
echo "Syncing dependencies..."
80+
tools/hab sync .
81+
82+
- name: Build
83+
run: |
84+
set +u
85+
source tools/envsetup.sh
86+
87+
echo "Generating build files for ${{ matrix.os_name }} ${{ matrix.arch_name }}..."
88+
89+
# Base args
90+
ARGS='enable_unittests=true enable_trace="perfetto" jsengine_type="quickjs" enable_frozen_mode=true use_sccache=true'
91+
92+
93+
# Platform specific args
94+
if [[ "${{ matrix.os_name }}" == "Darwin" ]]; then
95+
ARGS="$ARGS use_flutter_cxx=false"
96+
fi
97+
98+
# Note: If cross-compilation is needed for arm64 on Linux, target_cpu arg would be needed
99+
# But here we are running on native runners (macos-latest is arm64, others x64)
100+
101+
echo "GN Args: $ARGS"
102+
gn gen --args="$ARGS" out/Default
103+
104+
echo "Building benchx_cli..."
105+
ninja -C out/Default cli/benchx:benchx_cli
106+
107+
# Show sccache stats after build
108+
sccache -s
109+
110+
- name: Verify benchx_cli
111+
run: |
112+
# Create temp directory
113+
TEST_DIR=$(mktemp -d)
114+
echo "Using temp dir: $TEST_DIR"
115+
116+
# 1. Download test bundle
117+
curl -L -o "$TEST_DIR/GalleryComplete.lynx.bundle" https://unpkg.com/@lynx-example/gallery@0.6.7/dist/GalleryComplete.lynx.bundle
118+
119+
# 2. Run benchx_cli
120+
out/Default/benchx_cli -o "$TEST_DIR/GalleryComplete.ptrace" run "$TEST_DIR/GalleryComplete.lynx.bundle" --wait 1000
121+
122+
# 3. Download trace_processor
123+
# Download the downloader script
124+
curl -L -o "$TEST_DIR/trace_processor" https://get.perfetto.dev/trace_processor
125+
chmod +x "$TEST_DIR/trace_processor"
126+
127+
# 4. Verify trace content
128+
# Write SQL query to a file to handle multiple symbols cleanly
129+
# Using 'EOF' prevents shell expansion, making it safe for special characters
130+
cat > "$TEST_DIR/verify_symbols.sql" <<'EOF'
131+
SELECT name FROM slice WHERE
132+
name GLOB 'LynxEngine::LoadTemplate'
133+
OR name GLOB 'LynxShell::~LynxShell'
134+
OR name GLOB 'ReactLynx::diff::*'
135+
EOF
136+
137+
"$TEST_DIR/trace_processor" -q "$TEST_DIR/verify_symbols.sql" "$TEST_DIR/GalleryComplete.ptrace" > "$TEST_DIR/query_result.txt"
138+
139+
cat "$TEST_DIR/query_result.txt"
140+
141+
# Define expected symbols to check (one per line)
142+
cat > "$TEST_DIR/expected_symbols.txt" <<'EOF'
143+
LynxEngine::LoadTemplate
144+
LynxShell::~LynxShell
145+
ReactLynx::diff::
146+
EOF
147+
148+
FAILED=0
149+
while read -r SYMBOL; do
150+
[[ -z "$SYMBOL" ]] && continue
151+
if ! grep -qF "$SYMBOL" "$TEST_DIR/query_result.txt"; then
152+
echo "Error: Missing expected symbol '$SYMBOL' in trace output"
153+
FAILED=1
154+
fi
155+
done < "$TEST_DIR/expected_symbols.txt"
156+
157+
if [ "$FAILED" -ne 0 ]; then
158+
echo "Verification Failed: One or more expected symbols are missing."
159+
exit 1
160+
fi
161+
162+
echo "Verification Success: Found all expected symbols in trace."
163+
164+
# 5. Verify FiberElement Constructor/Destructor balance
165+
# Ensure every FiberElement::Constructor has a matching FiberElement::Destructor
166+
cat > "$TEST_DIR/verify_match.sql" <<'EOF'
167+
SELECT
168+
(SELECT COUNT(DISTINCT int_value) FROM args JOIN slice USING(arg_set_id)
169+
WHERE slice.name = 'FiberElement::Constructor' AND key = 'debug.id') as constructed_unique,
170+
(SELECT COUNT(DISTINCT int_value) FROM args JOIN slice USING(arg_set_id)
171+
WHERE slice.name = 'FiberElement::Destructor' AND key = 'debug.id') as destroyed_unique,
172+
(SELECT COUNT(*) FROM (
173+
SELECT int_value FROM args JOIN slice USING(arg_set_id)
174+
WHERE slice.name = 'FiberElement::Constructor' AND key = 'debug.id'
175+
EXCEPT
176+
SELECT int_value FROM args JOIN slice USING(arg_set_id)
177+
WHERE slice.name = 'FiberElement::Destructor' AND key = 'debug.id'
178+
)) as diff_count;
179+
EOF
180+
181+
"$TEST_DIR/trace_processor" -q "$TEST_DIR/verify_match.sql" "$TEST_DIR/GalleryComplete.ptrace" > "$TEST_DIR/match_result.txt"
182+
183+
cat "$TEST_DIR/match_result.txt"
184+
185+
# Extract the counts from the last line (format: "constructed","destroyed","diff")
186+
LAST_LINE=$(tail -n 1 "$TEST_DIR/match_result.txt")
187+
CONSTRUCTED=$(echo "$LAST_LINE" | cut -d',' -f1)
188+
DESTROYED=$(echo "$LAST_LINE" | cut -d',' -f2)
189+
DIFF=$(echo "$LAST_LINE" | cut -d',' -f3)
190+
191+
echo "Constructed: $CONSTRUCTED, Destroyed: $DESTROYED, Diff: $DIFF"
192+
193+
# Verify that we have some constructions, and diff is 0
194+
if [ "$DIFF" == "0" ] && [ "$CONSTRUCTED" -gt 0 ]; then
195+
echo "Verification Success: FiberElement Constructor/Destructor are balanced ($CONSTRUCTED objects)."
196+
else
197+
echo "Verification Failed: FiberElement imbalance or missing (Diff: $DIFF, Constructed: $CONSTRUCTED)."
198+
exit 1
199+
fi
200+
201+
# Cleanup
202+
rm -rf "$TEST_DIR"
203+
204+
- name: Prepare and Package Artifacts
205+
run: |
206+
mkdir -p dist/bin
207+
BINARY_PATH="out/Default/benchx_cli"
208+
209+
if [ ! -f "$BINARY_PATH" ]; then
210+
echo "Error: Binary not found at $BINARY_PATH"
211+
ls -R out/Default
212+
exit 1
213+
fi
214+
215+
echo "Found binary at $BINARY_PATH"
216+
cp "$BINARY_PATH" dist/bin/benchx_cli
217+
218+
# Create archive with specific naming format: benchx_cli_OS_Arch.tar.gz
219+
# e.g., benchx_cli_Darwin_arm64.tar.gz
220+
ARCHIVE_NAME="benchx_cli_${{ matrix.os_name }}_${{ matrix.arch_name }}.tar.gz"
221+
222+
cd dist/bin
223+
tar -czvf "../../$ARCHIVE_NAME" benchx_cli
224+
cd ../..
225+
226+
echo "Created archive: $ARCHIVE_NAME"
227+
228+
- name: Upload Artifact
229+
uses: actions/upload-artifact@v4
230+
with:
231+
name: build-artifact-${{ matrix.os_name }}-${{ matrix.arch_name }}
232+
path: benchx_cli_*.tar.gz
233+
234+
release:
235+
name: Create Release
236+
needs: build
237+
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/') && !inputs.skip_release
238+
runs-on: ubuntu-latest
239+
steps:
240+
- name: Download Artifacts
241+
uses: actions/download-artifact@v4
242+
with:
243+
path: artifacts
244+
merge-multiple: true # Merge all artifacts into one directory
245+
246+
- name: List Artifacts
247+
run: |
248+
ls -R artifacts
249+
250+
- name: Publish Release
251+
uses: softprops/action-gh-release@v2
252+
with:
253+
files: artifacts/*

‎.github/workflows/rebase_upstream.yml‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
name: Periodic Rebase
2+
3+
on:
4+
schedule:
5+
- cron: '0 0 * * *' # Daily at 00:00 UTC
6+
workflow_dispatch:
7+
8+
permissions:
9+
contents: write
10+
11+
jobs:
12+
prepare-rebase:
13+
runs-on: ubuntu-latest
14+
outputs:
15+
temp_branch: ${{ steps.create_branch.outputs.branch_name }}
16+
steps:
17+
- name: Checkout
18+
uses: actions/checkout@v4
19+
with:
20+
ref: develop
21+
fetch-depth: 0
22+
token: ${{ secrets.GITHUB_TOKEN }}
23+
24+
- name: Configure Git
25+
run: |
26+
git config --global user.name "github-actions[bot]"
27+
git config --global user.email "github-actions[bot]@users.noreply.github.com"
28+
29+
- name: Rebase Upstream and Push to Temp Branch
30+
id: create_branch
31+
run: |
32+
UPSTREAM_URL="https://github.com/lynx-family/lynx.git"
33+
34+
echo "Setting upstream to $UPSTREAM_URL"
35+
git remote add upstream "$UPSTREAM_URL"
36+
git fetch upstream
37+
38+
echo "Rebasing develop..."
39+
git checkout develop
40+
41+
if git rebase upstream/develop; then
42+
echo "Rebase successful."
43+
# Create a unique temporary branch name
44+
TEMP_BRANCH="rebase-check-${{ github.run_id }}"
45+
git checkout -b "$TEMP_BRANCH"
46+
git push origin "$TEMP_BRANCH" --force
47+
48+
echo "branch_name=$TEMP_BRANCH" >> "$GITHUB_OUTPUT"
49+
else
50+
echo "Rebase failed due to conflicts. Aborting."
51+
git rebase --abort
52+
exit 1
53+
fi
54+
55+
check-build:
56+
needs: prepare-rebase
57+
uses: ./.github/workflows/benchx_cli_publish.yml
58+
with:
59+
skip_release: true
60+
commit_ref: ${{ needs.prepare-rebase.outputs.temp_branch }}
61+
62+
push-to-develop:
63+
needs: [prepare-rebase, check-build]
64+
runs-on: ubuntu-latest
65+
if: success()
66+
steps:
67+
- name: Checkout Temp Branch
68+
uses: actions/checkout@v4
69+
with:
70+
ref: ${{ needs.prepare-rebase.outputs.temp_branch }}
71+
fetch-depth: 0
72+
token: ${{ secrets.GITHUB_TOKEN }}
73+
74+
- name: Push to Develop
75+
run: |
76+
git config --global user.name "github-actions[bot]"
77+
git config --global user.email "github-actions[bot]@users.noreply.github.com"
78+
79+
echo "Pushing verified rebase to develop..."
80+
git checkout develop
81+
# Reset develop to the verified temp branch commit
82+
git reset --hard origin/${{ needs.prepare-rebase.outputs.temp_branch }}
83+
git push origin develop --force-with-lease
84+
85+
# Cleanup
86+
git push origin --delete ${{ needs.prepare-rebase.outputs.temp_branch }}

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,3 +144,8 @@ js_libraries/lynx-core/src/common/feature.ts
144144
/tools/api/android
145145
/tools/api/ios
146146
/tools/api/docs/gen
147+
148+
compile_commands.json
149+
.cache/
150+
151+
.codspeed_*/

‎.habitat‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
habitat_version = "0.3.146-alpha.2"
1+
habitat_version = "0.3.146-alpha.4"
22
solutions = [
33
{
44
'name': '.',

‎BUILD.gn‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,3 +71,8 @@ group("third_party_trace_group") {
7171
deps += [ "base/trace/native:trace_tests" ]
7272
}
7373
}
74+
75+
group("cli") {
76+
deps = ["cli/benchx:benchx_cli"]
77+
testonly = true
78+
}

0 commit comments

Comments
 (0)