Skip to content

Commit f39e9aa

Browse files
zokrezylzokrezyl
committed
ni
1 parent d36bb3f commit f39e9aa

3 files changed

Lines changed: 735 additions & 0 deletions

File tree

‎tests/performance/git-yaafc/CMakeLists.txt‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# git-yaafc performance harness.
2+
#
3+
# Self-contained load generator: plain C + pthreads + POSIX sockets, no
4+
# yaafc libraries linked in (it speaks the gateway's HTTP contract, so
5+
# it also runs against the yaapp gateway). Built as `yaafc-perf`.
6+
find_package(Threads REQUIRED)
7+
8+
add_executable(yaafc-perf yaafc-perf.c)
9+
target_link_libraries(yaafc-perf PRIVATE Threads::Threads)
10+
set_target_properties(yaafc-perf PROPERTIES
11+
RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}")

‎tests/performance/git-yaafc/README.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# git-yaafc performance harness (`yaafc-perf`)
2+
3+
A load generator that drives the **gateway** (the production hot path) and
4+
reports latency percentiles + throughput under concurrency. It is the
5+
measurement counterpart to issue #4 §5.
6+
7+
## Design
8+
9+
- **Gateway only.** It never touches a backend port directly — same invariant
10+
as `mesh-up.sh`. Auth goes through `/login`; calls go through `/_rpc` or the
11+
HTML form routes, exactly as a real client would.
12+
- **Real threads, real sockets.** Each `--connections` worker is an OS thread
13+
with its own keep-alive TCP connection doing blocking request/response
14+
round-trips. Independent threads give honest concurrency and per-request
15+
latency without sharing — or being throttled by — the gateway's event loop.
16+
- **Self-contained.** Plain C + pthreads + POSIX sockets; links no yaafc
17+
libraries. The same binary therefore benchmarks the yaafc C gateway *or*
18+
yaapp's Python gateway — the HTTP contract is identical.
19+
20+
## Build & run
21+
22+
Built as part of the normal build:
23+
24+
```sh
25+
make build-desktop-release # produces build-desktop-release/yaafc-perf
26+
```
27+
28+
Bring the mesh up, then point the tool at it:
29+
30+
```sh
31+
./scenarios/git-yaafc/mesh-up.sh & # gateway on :8080
32+
./build-desktop-release/yaafc-perf --scenario rpc_count --connections 16 --duration 10
33+
```
34+
35+
### Options
36+
37+
| option | default | meaning |
38+
|---|---|---|
39+
| `--host H` | `127.0.0.1` | gateway host |
40+
| `--port P` | `8080` | gateway port |
41+
| `--connections N` | `8` | concurrent worker threads |
42+
| `--duration SECS` | `10` | run for this long |
43+
| `--requests R` | — | fixed requests **per worker** (overrides `--duration`) |
44+
| `--scenario NAME` | `rpc_count` | which workload (below) |
45+
46+
### Scenarios
47+
48+
| name | what it times (per iteration) | exercises |
49+
|---|---|---|
50+
| `rpc_count` | `POST /_rpc git_repo.store.count_total` | read; gateway → 1 backend |
51+
| `login` | `POST /login` | auth composite (4 backends) |
52+
| `repo_create` | `POST /repos/new` (authenticated) | write; storage + git_repo |
53+
| `register` | `POST /register` (new user each time) | write; accounts + authn |
54+
| `full` | register → login → repo create → list | end-to-end user journey |
55+
56+
Each worker establishes a session (register + login) before the timed loop for
57+
the scenarios that need one; `register`/`full` mint fresh users per iteration.
58+
59+
### Output
60+
61+
```
62+
scenario : rpc_count
63+
connections : 16 (16 sessions established)
64+
wall time : 4.004 s
65+
requests : 11226 ok, 0 errors
66+
throughput : 2803.6 req/s
67+
latency (ms) : mean=5.352 min=1.896 p50=4.064 p90=7.913 p99=31.250 p99.9=63.656 max=80.087
68+
```
69+
70+
Exit status is non-zero if any request errored or none succeeded — usable in CI
71+
gates once the runtime is fast enough to set thresholds. "sessions established"
72+
< connections is a red flag that results for a session-bound scenario are not
73+
trustworthy (the gateway/backends buckled under setup load).
74+
75+
## Extending
76+
77+
Add a `case` in `scenario_step()` and a row in `scenario_lookup()`'s
78+
`SCENARIOS` table. Keep each step one timed unit of work so the percentiles
79+
stay meaningful. Future additions: issues open/close, pipeline enqueue/lease,
80+
PAT mint, concurrent-session fan-out, delayed-backend stress.
81+
82+
## Early findings (baseline, blocking runtime — pre §5 async work)
83+
84+
Indicative local numbers at 16 connections (they vary run to run — treat as a
85+
baseline to beat, not a target):
86+
87+
| scenario | throughput | p50 | p99 |
88+
|---|---|---|---|
89+
| `repo_create` (write) | ~12k req/s | ~1.0 ms | ~3.5 ms |
90+
| `login` (auth, 4 backends) | ~590 req/s | ~22 ms | ~54 ms |
91+
| `register` (write) | ~100 req/s | ~136 ms | ~263 ms |
92+
93+
What stands out, and motivates the §5 async track:
94+
95+
- **`/_rpc` is round-trip heavy.** Each call costs ~6 *blocking* gateway→backend
96+
round-trips: opaque-token → uid resolution alone is `session` create + lookup
97+
+ destroy, then the call is object create + invoke + destroy. Obvious wins:
98+
cache the token→uid resolution, and reuse a per-session backend object instead
99+
of create/destroy per call.
100+
- **Auth composite (`login`) and `register` are an order of magnitude slower**
101+
than a single write — they serialise several backend round-trips on the
102+
blocking `rpc_call` path.
103+
- **The mesh is slow to recover from a load burst.** Right after a heavy run,
104+
session establishment for the *next* scenario can fail wholesale — the harness
105+
reports it honestly as "sessions established" dropping below `--connections`
106+
and then `0 requests` (it bails rather than spin). This is the head-of-line
107+
blocking / blocking-`rpc_call` / inline-storage behaviour §5 calls out. The
108+
harness inserts settle time between scenarios to give the mesh a chance to
109+
drain; the fact that it needs to is itself the finding.

0 commit comments

Comments
 (0)