Proposed fix of Issue #10064, rolling up nested cost trackers to the parent on closing

[asparagus]

Description

Rolling up nested cost trackers to the parent on closing

Context

This solves #10064

Caveats

Are any users relying on the current behavior, or already accounting for it?

Testing

Added tests/utils/test_usage_tracker.py::test_nested_usage_tracker, fixed in the current branch and errors on main with:

====================================================================================================== FAILURES =======================================================================================================
______________________________________________________________________________________________ test_nested_usage_tracker ______________________________________________________________________________________________

    def test_nested_usage_tracker():
        """Test that nested usage trackers still reflect usage on the parent."""

        def pretend_lm_call(prompt_tokens, completion_tokens):
            dspy.settings.usage_tracker.add_usage(
                "openai/gpt-4o-mini",
                {
                    "prompt_tokens": prompt_tokens,
                    "completion_tokens": completion_tokens,
                    "total_tokens": prompt_tokens + completion_tokens,
                }
            )

        def subagent(prompt_tokens, completion_tokens):
            with track_usage():  # subagent reports its own cost
                pretend_lm_call(prompt_tokens, completion_tokens)
            return prompt_tokens + completion_tokens

        with track_usage() as orchestrator_cost:
            pretend_lm_call(100, 10)
            subagent(1000, 100)
            subagent(500, 50)
            pretend_lm_call(5, 1)

        total_prompt_tokens = 100 + 1000 + 500 + 5
        total_completion_tokens = 10 + 100 + 50 + 1
        total_tokens = total_prompt_tokens + total_completion_tokens
>       assert orchestrator_cost.get_total_tokens()["openai/gpt-4o-mini"] == {
            "prompt_tokens": total_prompt_tokens,
            "completion_tokens": total_completion_tokens,
            "total_tokens": total_tokens,
        }
E       AssertionError: assert {'completion_..._tokens': 116} == {'completion_...tokens': 1766}
E
E         Differing items:
E         {'prompt_tokens': 105} != {'prompt_tokens': 1605}
E         {'completion_tokens': 11} != {'completion_tokens': 161}
E         {'total_tokens': 116} != {'...
E
E         ...Full output truncated (2 lines hidden), use '-vv' to show

tests/utils/test_usage_tracker.py:424: AssertionError
================================================================================================== warnings summary ===================================================================================================
tests/utils/test_settings.py::test_dspy_configure_allowance_async
  /Users/ariel/git/dspy-fork/tests/utils/test_settings.py:194: RuntimeWarning: coroutine 'test_dspy_configure_allowance_async.<locals>.foo4' was never awaited
    foo4()
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=============================================================================================== short test summary info ===============================================================================================
FAILED tests/utils/test_usage_tracker.py::test_nested_usage_tracker - AssertionError: assert {'completion_..._tokens': 116} == {'completion_...tokens': 1766}
================================================================================= 1 failed, 92 passed, 4 skipped, 1 warning in 21.27s =================================================================================

[greptile-apps]

Greptile Summary

This PR updates usage tracking so nested tracker contexts roll their usage into the parent tracker. The main changes are:

• Inner track_usage contexts now append their collected usage entries to the active parent tracker when they close.
• The parent tracker still remains unchanged when no parent tracker is active.
• A test was added for nested subagent usage so the outer tracker includes both direct and nested LM calls.

Confidence Score: 5/5

Safe to merge with minimal risk.

The change is small and localized to usage aggregation. The added test covers the reported nested-tracker behavior. No correctness or security issues were identified in the changed paths.

No files require special attention.

T-Rex Logs

What T-Rex did

• Reviewed the focused proof showing tests/utils/test_usage_tracker.py::test_nested_usage_tracker PASSED with EXIT_CODE 0.
• Reviewed the broader proof confirming all tests in tests/utils/test_usage_tracker.py completed with 8 passed, 1 skipped and EXIT_CODE 0.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
dspy/utils/usage_tracker.py	Updates track_usage to roll child tracker entries into an active parent tracker when the child context closes; no issues found.
tests/utils/test_usage_tracker.py	Adds test coverage for nested usage tracking so child LM usage is reflected in the outer tracker; no issues found.

_{Reviews (1): Last reviewed commit: "Proposed fix of Issue #10064, rolling up..." | Re-trigger Greptile}