This release adds support for Bodhi-less finalisation, Koji instance filtering, improved Bodhi timeout handling with email alerts, and stale Koji task cancellation on daemon restart.
-
Bodhi-less finalization (
bodhi: false). When the static config setsbodhito the literal valuefalse,RebuildBatch.run()now calls a newfinalize_via_tagging()path instead of going through Bodhi. This promotes draft builds, tags the NVRs directly into the stable tag, and waits for each NVR to appear using the configuredkoji.side_tag_timeout. Tagging failures are logged at ERROR level and trigger a failure email if the emailer is configured. -
Koji instance differentiation. Multiple Koji instances may share the same Fedora Messaging routing keys (e.g. the primary Koji and a secondary new-architecture instance). The daemon now checks
body.instancein incoming messages and filters for the configured instance, preventing cross-instance interference. Fixes #81. -
Bodhi timeout monitoring and email alerts. The daemon now sends a warning email when a Bodhi update is taking longer than expected and a failure email if the update tag wait times out entirely. Fixes #80.
-
Cancel stale Koji tasks on restart. On startup, the daemon now cancels any Koji tasks left running from a previous process, with a
--skip-stale-task-cancelCLI flag for local/personal Koji credentials and automatic skipping in dry-run mode.
- Added
koji.side_tag_timeoutconfig option, separating the initial side-tag creation wait from the Bodhi update tagging wait. - Made the Bodhi stable timeout configurable (default: 24 hours).
- Added validation checks for unlikely static config values.
- Added release and deployment documentation.
- Reordered Dockerfile
COPYlayer after package installs to improve layer caching and speed up build times. - Bumped version to 2.2.0; added James Troy as a project author.
This release represents a large body of work built on top of the 2.0.2
release, headlined by a full migration off Twisted onto native asyncio
and FastAPI, plus a hardened, fully-enforced build-pause feature and a
substantial new integration test suite.
- Twisted → asyncio/FastAPI migration.
web.pywas rewritten fromtwisted.webtoFastAPI/Starlette(withuvicorn),auth.pywas adapted to Starlette'sRequest/Response, and the daemon now runs the web app on the same sharedasyncioevent loop as the rest of the service.listener.py,db_models/buildtrigger, and the periodic config-update loop were all converted to be async-native:Deferred/@as_deferredbridging,twisted.internet.task.LoopingCall, anddeferToThreadwere replaced withasyncio.Future, a pure-asyncioPeriodicTask, andasyncio.to_thread, respectively.txrequestsandgetProcessOutputwere replaced withhttpx/asynciosubprocess calls, andpython-httpxwas subsequently swapped forpython-httpx2. Minimum supported Python is now 3.12. - Pause enforcement is now centralized and reliable. Build-pause
checking moved out of the message listener and web trigger endpoint and
into
batching.process_message_batch(), so no new Koji build can ever be started while paused. Incoming trigger messages,/triggerPOSTs, andperiodic_cleanup()all keep queuing/persisting work as normal while paused instead of being dropped or rejected — nothing is lost, and the message queue no longer backs up during a pause. A batch that is already running when a pause is requested is allowed to finish normally; only the next batch is held back until unpause. Extensive new integration coverage (tests/integration/test_pause_behavior.py) exercises pause toggled both via the static/dynamic config and via the runtime/control/pauseand/control/unpauseendpoints, including mid-batch pause timing. - New
bodhi.max_single_batch_sizeconfig option. Lets Bodhi update splitting (bodhi.batch_size) kick in only once a rebuild's total build count exceeds a separate threshold, so a mass-rebuild of many packages can be routed into smaller multi-build updates without also forcing small day-to-day rebuilds to be split. Defaults tobatch_sizeto preserve prior behavior when unset.
- Added a
/triggerrequest body JSON-schema validation step. /trigger's content-type parsing now correctly handles aContent-Typeheader that includes parameters (e.g.application/json; charset=utf-8).- Wired up periodic web session cleanup, which previously existed as unused code and let the active-session table grow without bound.
- Old/incomplete OIDC login states are now expired periodically to avoid unbounded memory growth and stale valid logins hanging around.
- Dropped support for untagging old package versions.
- Dropped the unused
kojihelpers.builds.wait_for_task()helper.
- Task registration now always happens before a wait begins, closing a race where a very fast Koji response could be delivered and ignored before the waiter was registered.
- Unknown/unexpected task errors are now handled as failures (populated
with synthetic timeout-like data) instead of raising
CancelledError, so a single bad task no longer aborts the whole batch. - Task cancellation now reliably surfaces as a task failure rather than as a cancellation.
pending_nvr_tagsentries are now cleaned up on timeout instead of leaking.- Fixed
Scheduling.reset(). - Ensured triggered builds finish submission before being considered handled.
git-lserrors are now handled instead of propagating unhandled.- The daemon now handles pod shutdown safely, and its shutdown
"terminator" was converted to an
asyncio.Future. - Fixed a "no running event loop" crash on daemon startup.
- Fixed a periodic config-update crash under the Twisted/asyncio reactor,
and
update_config()now catches unexpected errors so the periodic update loop keeps running afterward. ruff-driven cleanup ofdatetime.UTCusage.
- Added a large new end-to-end integration test suite
(
tests/integration/) with fake Koji, Bodhi, and Fedora Messaging bus implementations and a reusable test harness, covering full rebuild flows, task timeouts,check_tasks()polling, transient Koji submission errors, and pause behavior. - Added
tests/test_daemon.py,tests/test_listener.py,tests/test_scheduling.py,tests/test_state.py, and greatly expandedtests/test_web.pyandtests/test_parse_config.py. - Added
tox.inifor easy local test runs and simplifiedlocal_test_daemonconfiguration. - CI now runs pytest unit tests and dedicated integration-test jobs (using a virtualenv), with a corrected messaging configuration passed to the test daemon.
- Removed obsolete Twisted-era test scaffolding (
tests/deferredlist.py, Twisted-specific mocks) in favor of mockingcreate_subprocess_execand HTTP calls directly.
- Added
fastapianduvicorn. - Replaced
httpxwithhttpx2. - Removed
txrequests. - Raised minimum Python version to 3.12.