Skip to content

test: integration test robustness — stubs, retries, async, diagnostics - #597

Merged
scp7 merged 4 commits into
nolabs-ai:mainfrom
scp7:fix/tree-integration-test-stubs
Feb 2, 2026
Merged

test: integration test robustness — stubs, retries, async, diagnostics#597
scp7 merged 4 commits into
nolabs-ai:mainfrom
scp7:fix/tree-integration-test-stubs

Conversation

@scp7

@scp7 scp7 commented Feb 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fix missing DummyTreeTUI stub methods (advance_simple_progress, stop_live) that broke tree integration tests
  • Add pytest-rerunfailures and pytest-asyncio dependencies for flaky test retries and native async support
  • Replace asyncio.run() wrappers with native async def test methods (asyncio_mode = "auto")
  • Move @requires_openai/@requires_gemini skip markers to class level
  • Use shared openai_config fixture in tree tests (remove hardcoded provider constants)
  • Add assert_generation_result / assert_topic_build_result diagnostic helpers that surface failure details
  • Add @pytest.mark.flaky retries to all LLM-dependent integration tests
  • Suppress httpx "Event loop is closed" cleanup noise in conftest
  • Fix false-alarm "Topic paths (0) < expected samples" warning in generator when no topic model is used
  • Improve make test-integration-verbose with -s, --tb=long, --durations

Test plan

  • make lint passes
  • make test (unit tests) passes — 528 tests
  • make test-integration with OPENAI_API_KEY set — all OpenAI tests pass
  • Integration tests collect correctly (25 tests across 5 files)
  • No "Topic paths (0)" false-alarm warnings without topic model
  • No "Event loop is closed" noise in verbose output

DummyTreeTUI was missing advance_simple_progress() and stop_live()
methods that _process_tree_events in topic_manager now calls.
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @scp7, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves an issue in the integration test suite by updating a mock object to include methods that were recently added to the production code. This ensures that the tests accurately reflect the current interface and functionality, preventing potential test failures due to incomplete mock implementations and maintaining the reliability of the test suite.

Highlights

  • Test Fix: Addresses a bug in the DummyTreeTUI mock object within test_tree_integration.py where two essential stub methods, advance_simple_progress() and stop_live(), were missing.
  • Root Cause: These missing methods are called by _process_tree_events in topic_manager.py, a change introduced in a previous pull request (feat: UUID-based cycle generation with bug fixes and UX improvements #595), which was not reflected in the test mock.
  • Resolution: The pull request adds these pass-through stub methods to DummyTreeTUI to ensure the integration tests correctly simulate the expected behavior without actual implementation.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly adds missing stub methods, advance_simple_progress() and stop_live(), to the DummyTreeTUI mock class in an integration test, resolving a crash. The change is straightforward and effective. I've included one suggestion to further improve the test's robustness by not only stubbing one of the new methods but also verifying that it's called, which would provide stronger guarantees about the component's behavior.

Comment thread tests/integration/test_tree_integration.py
scp7 added 2 commits February 2, 2026 13:34
- Add pytest-rerunfailures and pytest-asyncio dependencies
- Configure asyncio_mode=auto for native async test support
- Replace asyncio.run() wrappers with native async test methods
- Move @requires_openai/@requires_gemini to class level
- Use shared openai_config fixture in tree tests (remove hardcoded constants)
- Add assert_generation_result/assert_topic_build_result diagnostic helpers
- Add @pytest.mark.flaky retries to all LLM-dependent tests
- Suppress httpx "Event loop is closed" cleanup noise in conftest
- Fix false-alarm "Topic paths (0) < expected samples" warning in generator
- Improve make test-integration-verbose with -s, --tb=long, --durations
@scp7 scp7 changed the title fix: add missing DummyTreeTUI stub methods in integration test test: integration test robustness — stubs, retries, async, diagnostics Feb 2, 2026
@scp7

scp7 commented Feb 2, 2026

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly improves the robustness and maintainability of the integration test suite. Key changes include adding pytest-rerunfailures for flaky tests, converting synchronous asyncio.run calls to native async def tests, and introducing detailed diagnostic assertion helpers. The refactoring to use shared fixtures and class-level markers also enhances code clarity. Overall, these are excellent improvements. I've added a couple of minor suggestions to further refine the new helper functions.

Comment thread tests/integration/conftest.py Outdated

@pytest.fixture
def tree_builder(self):
def tree_builder(self, openai_config):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The self parameter is not used in the tree_builder fixture. To make it clear that this fixture doesn't depend on the test class instance, you could either move it outside the class or declare it as a static method (which would also involve removing the self parameter).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ignored.

@scp7
scp7 merged commit 9c1d9f4 into nolabs-ai:main Feb 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant