fix(maf): prevent metric inflation and context breaks

[sipercai]

Description

This PR fixes two Microsoft Agent Framework observability regressions within the MAF instrumentation only:

• removes the process-cumulative ObservableGauge callbacks from MAFSemanticProcessor, allowing MAF's native counter/histogram instruments to remain the single metric source;
• routes MAF's tracer and meter access through the explicitly supplied providers and keeps the MAF-owned span current for non-streaming work, restoring the expected parent/trace relationship and cleaning up context when the call returns.

When the optional Robin SDK extension is present, the bridge also scopes its LLM SDK suppression key to the active MAF LLM call so the underlying SDK does not emit a second LLM span.

The shared util/opentelemetry-util-genai package is intentionally unchanged. Existing MAF constructor/config compatibility surfaces remain available, but no longer register or emit the removed cumulative gauges.

Fixes # (N/A)

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature which causes existing functionality not to work as expected)
• This change requires a documentation update

How Has This Been Tested?

• MAF latest and oldest tox suites
• Base/head minimal regression probe
• Live non-streaming, streaming, multi-turn, concurrency, tool/ReAct, and error-path smokes
• ARMS/CMS trace and metrics readback
• Weaver OTLP live-check
• Fresh-cache repository pre-commit and typecheck

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist

• Followed the style guidelines of this project
• Changelog has been updated
• Unit tests have been added or updated
• Documentation has been updated

Validation Evidence

Scope

• Changed package: loongsuite-instrumentation-microsoft-agent-framework only.
• Shared util change: none (git diff origin/main...HEAD -- util/opentelemetry-util-genai is empty).
• CI package detector: |loongsuite-instrumentation-microsoft-agent-framework|; not degraded and not full-repo.

Fix Verification

The same isolated repro was run on merge base be4f7d3 and on this branch:

Check	Base	Head
MAF ObservableGauge registrations	6	0
Exported spans with explicit providers	0	2
Non-streaming span shares the outer trace	no	yes
Non-streaming span is parented to the outer span	no	yes

Local Checks

Check	Command	Result
Static readiness	python "$PIPELINE_SKILL_DIR/scripts/check_loongsuite_pr_readiness.py" --repo .	pass
Fresh precommit	PRE_COMMIT_HOME="$(mktemp -d /tmp/loongsuite-precommit.XXXXXX)" tox -e precommit	pass
Typecheck	tox -e typecheck	pass, 0 errors / 0 warnings
MAF latest	tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-microsoft-agent-framework	72 passed
MAF oldest	tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-microsoft-agent-framework-oldest	72 passed
MAF lint	tox -c tox-loongsuite.ini -e lint-loongsuite-instrumentation-microsoft-agent-framework	pass
Privacy / diff checks	focused diff scan and git diff --check origin/main...HEAD	pass
External review	deep review loop	no blocking findings; one P3 compatibility-helper cleanup suggestion deferred to keep this fix minimal

Real E2E Matrix

Scenario	Status	Evidence
non-streaming	pass	two consecutive calls produced separate traces; each restored its legitimate outer context and left no active span after the outer scope exited
streaming	pass	completed span tree and clean context after iteration
multi-turn	pass	history was preserved and the two turns remained independently rooted
concurrency	pass	two concurrent calls produced isolated trace IDs with no cross-trace parents
agent/tool/ReAct	pass	one trace contained 1 AGENT, 2 STEP, 2 LLM, and 2 TOOL spans with AGENT -> STEP -> LLM + TOOL
error path	pass	the expected exception produced ERROR spans without leaving stale context

The cloud readback found all 9 scenario trace IDs and all 39 emitted spans. The canonical GenAI trace validator passed all 7 GenAI spans in the two-tool ReAct trace.

Metrics

The live SDK run emitted only MAF's native metrics:

• gen_ai.client.operation.duration
• gen_ai.client.token.usage
• agent_framework.function.invocation.duration

No legacy genai_calls_* or genai_llm_usage_tokens SDK gauge was created. The backend-derived call metric readback contained exactly 30 GenAI calls, matching the 30 GenAI spans in the trace readback, and had only the backend xtrace source.

Telemetry Contract

Check	Status	Notes
Span tree / parent integrity	pass	no broken or cross-trace parents
NO_CONTENT	pass	zero content attributes
SPAN_ONLY	pass	expected span content attributes were present
SPAN_AND_EVENT	N/A	MAF currently captures content on spans and does not emit GenAI content events
Weaver OTLP live-check	pass with known registry advisories	all target gen_ai.* attributes were recognized; remaining violations are existing registry gaps for standard resource attributes and native MAF tool-duration attributes

Existing CI Context

The current remote checks belong to the previous broader commit that also changed the shared util package. That scope caused the package detector to run all util consumers. The Google ADK failures are missing orjson, and the LiteLLM failures are missing fastapi; both reproduce as dependency-environment failures and are unrelated to the MAF change. After this MAF-only revision is pushed, the detector selects only the MAF package.

[ralf0131]

Summary

Fixes two MAF observability regressions: (1) removes process-cumulative ObservableGauge callbacks from MAFSemanticProcessor that caused GenAI metric inflation, routing MAF's native metric instruments to explicitly supplied providers via util_genai_bridge; (2) adds a generic existing-span lifecycle API (use_existing_span, mark_first_token, record_existing_span) to opentelemetry-util-genai so MAF-owned AGENT spans remain current without transferring span ownership or ending native spans. Includes thread-safe locking and fork-safe lock reinitialization.

Overall verdict: looks good. Well-structured, comprehensive test coverage (thread safety, fork safety, legacy constructor compat, streaming, concurrent finalization, error paths). The architectural change of moving metrics aggregation out of the span processor and into the util-genai handler is the right approach — it eliminates the ObservableGauge inflation at the source.

Posting as COMMENT rather than APPROVE because CI is still pending (PR was pushed ~20 min ago). Will approve once CI passes.

Findings

• [Info] extended_handler.py:223 — isinstance guard replaced with cast() (duck typing); subtle behavior change for custom recorders
• [Info] __init__.py:136 — Good defensive rollback on add_span_processor failure

Suggestions

• Consider documenting the record_extended signature change (record_standard_metrics parameter) and the isinstance→cast switch as a breaking change for custom ExtendedInvocationMetricsRecorder subclasses in the changelog, since downstream instrumentations that vendor a custom recorder could be affected.

Cross-repo Note

Changes to util/opentelemetry-util-genai (existing-span lifecycle API, record_extended signature, _prepare_model_context hook) may affect downstream instrumentation plugins that depend on this util package. The related repos (loongsuite-pilot, loongcollector) don't directly import the Python util, so no immediate cross-repo action needed.

---

Automated review by github-manager-bot

[ralf0131]

[Info] Good defensive rollback — if add_span_processor fails, calling revert_util_genai_bridge() and _remove_span_processor() prevents leaving the instrumentation in a partial state where the bridge is installed but the processor is not. Solid error handling.

[ralf0131]

Summary

MAF instrumentation fix for metric inflation and context breaks. Scope narrowed to MAF-only since the previous revision (shared util/opentelemetry-util-genai changes removed). CI now passes on all core quality checks.

Previous review (commit 6028e076) was COMMENT-only pending CI; my prior findings about the util/opentelemetry-util-genai existing-span lifecycle API no longer apply since those changes were removed. The remaining MAF-only changes are sound:

• Removing the process-cumulative ObservableGauge callbacks eliminates metric inflation at the source — MAF's native counter/histogram instruments remain the single metric source.
• Routing through explicitly supplied tracer_provider/meter_provider via util_genai_bridge restores correct parent/child trace relationships.
• Non-streaming span activation + optional Robin SDK suppression is properly guarded (ImportError fallback) and scoped to the active LLM call.
• Rollback via revert_util_genai_bridge() on add_span_processor failure is a good defensive measure.
• Test coverage is comprehensive (no-gauge registration, rollback, span context + suppression, provider routing).

LGTM — approving now that CI passes and the scope is contained to the MAF package.

---

Automated review by github-manager-bot