Repository navigation
ch5 registry/CPU gauge, ch7 error.type, stable HTTP semconv opt-in, Linux scraping (ch3-11) - #71
Conversation
… uids, enable exemplars
…m.cpu.utilization gauge, type and test both services
…on Linux (ch3-11)
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…ery Grafana rule is healthy; ch8 README histogram name
# Conflicts: # .github/workflows/ci.yml
…enders OpenMetrics
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 85adc47a8b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| import httpx | ||
| import pytest | ||
| from fastapi.testclient import TestClient |
There was a problem hiding this comment.
Handle the TestClient deprecation under CI's warning policy
With the locked Starlette version, these new TestClient-based tests emit a deprecation warning, as also shown in the commit's test evidence. However, the Python job in .github/workflows/ci.yml runs uv run pytest -W error, so that warning becomes an error and the newly enabled chapter 5 test suite fails CI even though ordinary pytest passes. Use the supported client API or narrowly filter this dependency warning while retaining the existing warning policy.
Useful? React with 👍 / 👎.
|
@soumendrak-reviewer-bot review |
There was a problem hiding this comment.
REJECT at f3f7ad41099d295b93b6310a2b1b4863b9a5b05c. CRITICAL (edits .github/workflows/ci.yml). Round 3. Scope: the delta 1a67a0f..f3f7ad4 (merge c952641, dfc0cc7, 85adc47, f3f7ad4) plus regressions, and one finding I missed earlier, labelled as such.
CURRENT STATE. Everything from the earlier rounds is fixed, and I re-ran it at this head:
- Merge
c952641has exactly the tree of a fresh merge of1a67a0fandbc3d49b(957ab0f). Nothing extra came in with it. - The ch5 regression from
85adc47is gone. The CIpythonjob body, run locally for all 11 chapters, exits 0 on every one. ch5 underpytest -W errorgave5 passed10 times out of 10, withapi_gateway.pyandorder_service.pyat 100%.mypy --strictis clean anduv lock --checkpasses. - ch7: the
ignore_errorsoverride is gone,mypy --strictreportsSuccess: no issues found in 3 source files, and the run gives13 passed. The 5 mutants (error.typeon the fallback,ERRORstatus on the fallback, droppingrecord_exception, renaming thecache_fallbackevent, and plain-text/metrics) are all killed. The CONTROL passes. - CI has 16 of 16 check runs green. The Codex P2 is fixed.
One defect is still there, in the except block this PR edits: every failed Stripe call records its exception twice.
EXPECTED STATE. Each failed external.stripe.charge span carries exactly one exception event, a test fails if a second one comes back, and CI is green at the new head.
Must-fix
- MISSED EARLIER FINDING (it was in the round-1 diff):
call_external_apirecords the exception twice. Location:chapters/ch7-error-handling/order_service.py:177-198. The manualspan.record_exception(e)is at :191, followed by a re-raiseat :195.start_as_current_spankeeps its defaultrecord_exception=True.- Proof: drive
create_orderthrough an in-memory exporter, with the DB stubbed OK and the Stripe charge raisingConnectionError. The result isexternal.stripe.charge StatusCode.ERROR ['exception', 'exception'].insert_order_recordhas['exception']. The current 13 tests still pass, so nothing guards this. The PR body reports the same thing from Jaeger: 8 of 8 failures had two events. - WHY: the SDK's context manager records any exception that leaves the
withblock. Recording it by hand and then re-raising stores it a second time. A chapter whose README saysrecord_exception"adds anexceptionEvent" then shows learners two identical stack traces on every failed call, and doubles the event volume. That teaches the wrong lesson in the one chapter about error recording. This PR is the one editing that block forerror.type, so deferring it leaves a known defect in code it changes. - CHECK: add a test asserting
[e.name for e in span_named("external.stripe.charge").events] == ["exception"]. It must fail atf3f7ad4. Then fix the double recording: either passrecord_exception=Falseto thatstart_as_current_span, or drop the manual call. I checked the first option: the probe prints['exception']and all 13 tests still pass. The test must pass after the fix.
- Proof: drive
Merge note: squash this PR, because 8fe987d (six .coverage files) is in its history. #70 is still REJECT at bc3d49b, so after #70 changes, this branch needs it merged in again and the verdict re-checked.
Any new commit voids this verdict.
…he CLIENT span in ch7-ch11
soumendrak-reviewer-bot
left a comment
There was a problem hiding this comment.
APPROVE @ cfafaba (round 4, CRITICAL: ci.yml)
Prior must-fix FIXED:
- Commit 19a8918 fails in ch7-ch11 with
['exception','exception'] == ['exception'], and cfafaba passes (ch7 15/15, ch8-ch11 12/12 each). - An in-memory exporter, run 3 times per chapter, shows exactly one exception event per failure, with status ERROR and description "ConnectionError: stripe API timeout".
- An AST scan finds no other record-and-reraise inside a span in ch7-ch11.
The claim that the "set_status dropped" mutant is equivalent is confirmed in two ways:
- A differential run gives identical output in all five chapters.
- In OTel 1.45.2, use_span records the exception and sets ERROR on escape (api trace/init.py L609-628), and set_status lets ERROR overwrite ERROR (sdk trace/init.py L981-1000).
The "success path never sets status" line (order_service.py:186-189) dates from 490846c and was not touched by this PR, so it is out of round scope.
mypy --locked passes for ch5-ch11, the README matches the behaviour, the size reproduces at +177/-125, and CI is 16/16. Squash-merge, after #70 is merged.
Stacked on #70 (base
fix/json-logging). Merge order: #67 → #68 → #69 → #70 → this.What changed (audit items)
REGISTRY.register(reader)fromapi_gateway.pyandorder_service.py.PrometheusMetricReader.__init__already registers its own collector, so the extra call raisedTypeError: 'NoneType' object is not iterableon every start, and thetry/excepthid it.setup_metrics()now just returns theMeterProvider.system.cpu.utilizationObservableGauge. It uses unit1and reportspsutil.cpu_percent() / 100, a 0–1 ratio, per the system metrics semconv. The article uses%, so the article PR has to match this. psutil was already a dependency.mypy --strictpasses. pytest checks 100% coverage ofapi_gateway.pyandorder_service.py. Dev deps: pytest, pytest-cov, mypy, types-psutil.error.typeon failed spans, following Recording errors: "When the operation ends with an error, instrumentation … SHOULD set theerror.typeattribute".order_service.py,insert_order_recordnow sets it on DB and payment failures.call_external_apisetserror.typewhere it used to setexternal.error_type.error.type, as the spec requires for handled errors.error.typeto the status code there ("500","502", seen in Jaeger below). A manual value would just be overwritten.export OTEL_SEMCONV_STABILITY_OPT_IN := http, so auto-instrumented spans use the stable HTTP names (HTTP semconv migration). Checked for impact first:http.targetandurl.path.http_*metric labels, which this doesn't touch.down, which breaks item 8's Prometheus checks.host.docker.internaldoesn't resolve on Linux withoutextra_hosts: host-gateway.127.0.0.1by default (settings).extra_hoststo the ch5–7 Prometheus service and--host 0.0.0.0torun-order/run-gateway. Docker Desktop already resolveshost.docker.internal, so nothing changes there.context.get_current()passed to.record(). They actually come fromprometheus_client'sHistogram.observe(..., exemplar=...), and the README now says so (item 2).Skills applied
engineering-rules-for-soumen-s-repos, verify-changes-before-review, tdd (tests failed on the old code first), principle-prove-it-works, principle-fix-root-causes, principle-subtract-before-you-add (dropped the dead registration and the gateway
error.typelines that the instrumentation overwrites), blast-radius (grep of every config, dashboard and alert before the semconv opt-in), no-comments, sourced-content-claims, pr-queue-hygiene.Codex
Not used.
Evidence
New ch5 tests against the old services:
New ch7 span tests against the old
order_service.py:After the change, with 10 repeated runs each:
(Round 3: that Starlette
httpxTestClient deprecation broke CI underpytest -W erroronce ch5 had tests; fixed in f3f7ad4, see below.)ch3 spans in Jaeger,
make run-zerowith and without the opt-in (/api/v3/traces, HTTP keys only):ch5/6/7 stacks:
docker compose up --wait, thenmake run-order run-gateway, then 20×/checkout. Before the host fix:After:
ch7 error spans in Jaeger after 20 checkouts, as (span, status, error.type, has external.error_type):
302 changed lines (+177/-125) against
fix/json-logging, excluding tests anduv.lock(round 4 head cfafaba;git diff --numstat origin/fix/json-logging...HEAD). The same count againstmain+ #70 merged: +177/-125.Review round 3 (heads 85adc47, f3f7ad4)
Fixed with normal commits on top of a normal merge of
fix/json-logging(c952641). Nothing was force-pushed or rebased.test_cache_fallback_span_is_not_an_error: sourcecache, status UNSET, noerror.type,product.source=cache, events[exception, cache_fallback]ignore_errorsoverride and fix the typespyproject.toml;order_service.pyfully typed (response_model=Noneon the union-returning/orders, a typedrender_openmetricsboundary for the untypedprometheus_client.openmetricsgenerate_latest). Noignore_errors,type: ignoreornoqaleft in any chapter/metricscoverage after the retypetest_metrics_endpoint_renders_openmetrics-W errorhttpx2>=2.16.0to ch5 dev deps, same as ch4 in #68. CI on 85adc47 failed exactly this way (ModuleNotFoundError: No module named 'httpx2'→StarletteDeprecationWarning)Mutation check (ch7 @ f3f7ad4,
uv run pytest -W error)ch7
mypy --strict:Success: no issues found in 3 source fileswith the override gone. ch5 @ f3f7ad4:5 passed, coverage 100%, mypy strict clean, ruff check clean, underpytest -W error.Live ch7
error.type(stack @ 85adc47, Jaeger/api/v3/traces, order-service stopped for the last requests)/checkout→/orderspayment failure./products/{id}only returns 503 (order-service unreachable), and its server span carrieserror.type="503".Finding, not fixed here: stripe doubleFixed in round 4 (cfafaba), see below.exceptioneventOTEL_SEMCONV_STABILITY_OPT_IN=http, auto-instrumentation already setserror.typeto the status code on every 5xx server span. A FastAPI TestClient check confirms it overwrites a manually set value (Custom→"500"), and marks the route span ERROR whether the handler raises or returnsJSONResponse(500). Onlyraiseadds anexceptionevent to the route span.Boots @ 85adc47 (
docker compose up,make run-order run-gateway, traffic)Still open (not in this PR)
error.typefollow-up: the sameinsert_order_record/call_external_apierror.typechanges for ch8–11. (The stripe double-exception fix for ch8–11 is now in this PR.)ruff format --checkwants to wrap one unchanged line in ch5api_gateway.py(request_counter.add(0, ...)). CI does not run ruff format.Skills applied (round 3)
learn-from-review-rejections, engineering-rules-for-soumen-s-repos, verify-changes-before-review, principle-prove-it-works, principle-fix-root-causes, principle-test-behavior-not-implementation, principle-type-system-discipline, no-comments, pr-queue-hygiene, pre-delivery-checks.
Codex
Not used. The
chatgpt-codex-connectorreview bot comments on this repo by itself; I fixed its finding and did not use Codex to write anything.Review round 4 (rejected at f3f7ad4: stripe double
exceptionevent) → head cfafabaNew commits only. Nothing was force-pushed or rebased. Commits:
origin/fix/json-logging(JSON logs via json.dumps: escape messages, keep keyword fields (ch6-11) #70 @ 8955cfe), clean, so the round-2 JSON logs via json.dumps: escape messages, keep keyword fields (ch6-11) #70 test and the CI, Python 3.13 per chapter, promtool test fix, docker compose v2 #67/Bump OTel to 1.45.2/0.66b2, pin collector 0.164.0, Jaeger 2.22, Prometheus 3.15.1, Grafana 13.3 #68 fixes are in this branch.['exception']on aConnectionError, failing at f3f7ad4test_failed_payment_records_one_exception_event(event names== ["exception"],exception.type == "ConnectionError"). ch8–11: newtests/test_external_call.py(byte-identical in all four), withtest_failed_external_call_records_one_exception_event(events== ["exception"],exception.type, status ERROR) andtest_successful_external_call_records_no_eventsrecord_exception=Falseor drop the manual call, whichever matches the READMErecord_exception=Falseon theexternal.<service>.<op>CLIENT span incall_external_api. The README teaches the manualspan.record_exception(e)and "an exception Event", so the manual call stays and the SDK's automatic second recording goesstart_as_current_spanwithoutrecord_exception=Falsewhose body callsspan.record_exceptionand re-raises): found onlycall_external_apiin ch7, ch8, ch9, ch10 and ch11. No open PR other than this one touches thoseorder_service.pyfiles (#69 is merged, #70 is logging-only), so all five are fixed here. After the fix the scan finds nothingevents: exception (exactly one), plus a Key Concepts row "Record once, then re-raise" that explains whyrecord_exception=Falseis therech8–11
order_service.pywere untyped. Importing them from the new tests pulled them into CI'smypy --strict, so they are now fully annotated:dict[str, Any]parameters andCallable[..., Awaitable[Any]];response_model=Noneon the union-returning/ordersand/ask, the same as ch7.Tests fail first
At 19a8918 (tests only, the f3f7ad4 code):
At cfafaba: ch7 15 passed, ch8–11 12 passed each. Coverage is 100% and
mypy --strictis clean in all five.ruff checkreports nothing new compared with f3f7ad4.Mutation check (cfafaba,
PYTHONDONTWRITEBYTECODE=1,__pycache__cleared before each run)record_exception=Falseremoved (double event)record_exception=True(double event)span.record_exceptiondropped as well (zero events)span.set_status(ERROR)droppedset_statusdropped is an equivalent mutant.set_status_on_exception=True(the SDK default) sets ERROR anyway when the re-raised error leaves thewithblock, so no test can tell them apart.test_successful_external_call_records_no_events.Live ch7 stack (Jaeger
/api/v3/traces, events per span)f3f7ad4-equivalent code (85adc47):
cfafaba:
Follow-up, not fixed here
set_status_on_exception=True, the SDK replaces the manual status description (stripe API timeout) withConnectionError: stripe API timeoutwhen the error leaves the span. That is harmless, so it is left as is. It could be changed withset_status_on_exception=Falseif the manual description matters.Skills applied (round 4)
learn-from-review-rejections, tdd (tests committed and shown failing before the fix), engineering-rules-for-soumen-s-repos, verify-changes-before-review, principle-prove-it-works, principle-fix-root-causes, principle-test-behavior-not-implementation, principle-type-system-discipline, no-comments, pr-queue-hygiene, pre-delivery-checks.
Codex NOT used.
Checked: at cfafaba, the new tests fail at 19a8918 and pass at cfafaba in ch7–11; the AST scan finds no double-record left; the event-count mutants are killed in all five chapters; the live ch7 stripe ERROR spans carry one
exceptionevent;git grepfinds no noqa, type: ignore, pragma: no cover or ignore_errors; CI on cfafaba is green (16/16 checks pass).