Skip to content

Live API healthcheck and test updates#151

Open
DoraFgr wants to merge 7 commits intolichess-org:masterfrom
DoraFgr:feature/workflow-live-api-healthcheck
Open

Live API healthcheck and test updates#151
DoraFgr wants to merge 7 commits intolichess-org:masterfrom
DoraFgr:feature/workflow-live-api-healthcheck

Conversation

@DoraFgr
Copy link
Copy Markdown
Contributor

@DoraFgr DoraFgr commented Dec 6, 2025

PR: Live API healthcheck & opening-explorer typing fixes

This PR implements issue #65.

Goal

  • Add a live-API healthcheck workflow that runs the test-suite against the live Lichess API and files an issue automatically when failures are detected.

What changed

  • Tests & marker

    • Introduced a new pytest marker streaming to mark tests that rely on streaming endpoints.
    • Streaming tests are excluded from the weekly live healthcheck to avoid very long workflow runs.
  • Workflow & tooling

    • Added a GitHub Actions workflow live-api-healthcheck to run tests against the live API on a schedule and on-demand.
    • Added a Makefile target test_live_no_streaming to run the live tests while skipping streaming-marked tests. The workflow uses this target so the scheduled run completes reliably.
  • Types and client

    • Improved typing for the opening-explorer client by adding new typed dicts: PlayerOpeningStatistic and MastersOpeningStatistic (in berserk/types/opening_explorer.py).
    • Updated client.opening_explorer method signatures and the tests to use the new types.

Workflow test:

To validate the workflow behavior three scenarios were tested:

  1. Workflow where live tests pass against the live Lichess API (no issue created).
  2. Workflow where live tests fail and an issue is automatically created describing the failing run.
  3. Workflow where live tests fail but an issue is not created because a matching open issue already exists.

These validate that the healthcheck detects regressions, avoids duplicate issues, and stays actionable.

Note

  • Streaming endpoints can intermittently hang or take many minutes to finish. Running them in a scheduled healthcheck makes the workflow unstable or slow. therefore the streaming-marked tests are excluded from the weekly run but kept available for ad-hoc troubleshooting and local runs.

How to run locally

  • Run live tests but skip streaming-marked tests:
make test_live_no_stream

Checklist when adding a new endpoint
  • Added new endpoint to the README.md
  • Ensured that my endpoint name does not repeat the name of the client. Wrong: client.users.get_user(), Correct: client.users.get()
  • Typed the returned JSON using TypedDicts in berserk/types/, example
  • Written tests for GET endpoints not requiring authentification. Documentation, example
  • Added the endpoint and your name to CHANGELOG.md in the To be released section (to be created if necessary)

- Introduced GitHub Actions workflow for live API healthcheck to validate API changes.
- Added `test_live_api` target in Makefile to run tests against the live API without cassettes.
- Implemented `--live-api-throttle` option in pytest for throttling between live API tests.
@DoraFgr DoraFgr marked this pull request as ready for review December 6, 2025 16:08
# The id of the OTB master game
id: str
# The winner of the game. Draw if None
winner: Literal["white"] | Literal["black"] | None
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably rather not required Color from common

# The year of the game
year: int
# The month and year of the game. For example "2023-06"
month: NotRequired[str]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this would be not required

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am using this schemas doc as a reference, is there another reference I should be looking at?

Comment on lines 84 to +86
game: GameWithoutUci | None
# The opening info for this move
opening: Opening | None
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most likely not required

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this schema doc have it has required.

Comment on lines +105 to +107
game: GameWithoutUci | None
# The opening info for this move
opening: Opening | None
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this schema doc have it has required.

Comment on lines +124 to +126
game: MastersGameWithoutUci | None
# The opening info for this move
opening: Opening | None
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this schema doc have it has required.

Comment on lines +146 to +164
class PlayerOpeningStatistic(TypedDict):
# Number of game won by white from this position
white: int
# Number of game won by black from this position
draws: int
# Number draws from this position
black: int
# Opening info of this position
opening: Opening | None
# The list of moves played by the player from this position
moves: List[PlayerMove]
# recent games with this opening
recentGames: List[Game]
# Queue position for indexing (present when wait_for_indexing parameter used)
queuePosition: NotRequired[int]


class MastersOpeningStatistic(TypedDict):
# Number of game won by white from this position
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be refactored with generic typeddict https://alexocallaghan.com/python-typeddict-with-generics

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good idea! I did not knew about generic type. Thanks!

@DoraFgr DoraFgr force-pushed the feature/workflow-live-api-healthcheck branch from e3cdf92 to 4daf82a Compare December 8, 2025 00:08
…formation to match current Lichess API schema.
- Updated `TestMasterGames` and `TestPlayerGames` to validate responses against typed-dicts `MastersOpeningStatistic` and `PlayerOpeningStatistic`.
- Removed hardcoded assertions for response values in favor of validation utility.
- Simplified the `test_stream` method to ensure at least one result is yielded from the stream.
@DoraFgr DoraFgr force-pushed the feature/workflow-live-api-healthcheck branch from 4daf82a to c369abb Compare December 8, 2025 00:13
@DoraFgr
Copy link
Copy Markdown
Contributor Author

DoraFgr commented Dec 23, 2025

Hi @kraktus, just following up on this PR. Let me know if there's anything else you'd like me to adjust or clarify.

Thanks!

@DoraFgr
Copy link
Copy Markdown
Contributor Author

DoraFgr commented Feb 27, 2026

Hi @kraktus, just following up on this PR. As mentionned I try to schema doc from the API as reference when determining the type of object. Let me know if there's anything else you'd like me to adjust or clarify.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants