NOJIRA-Sandbox-install-reliability-sentinel-stt-fix - #1147
Merged
pchero merged 3 commits intoJul 30, 2026
Conversation
call-manager and timeline-manager both include the sentinel-manager event exchange in their subscribe-target lists, but sentinel-manager requires the Kubernetes API and is only deployed in Kubernetes environments. In any other deployment nothing declares that exchange, so QueueSubscribe's underlying QueueBind fails with an AMQP 404, which closes the channel shared by all of the queue's bindings and makes the service exit fatally at boot. Both services now declare the exchange themselves immediately before that one target's bind, using the same durable fanout parameters sentinel-manager's own notifyhandler uses, so the declare is an idempotent no-op when sentinel-manager is deployed. - bin-call-manager: Call sockHandler.TopicCreate for the sentinel target in subscribeHandler.Run() before its QueueSubscribe; return the error on failure, matching the existing subscribe-failure handling in the same loop - bin-call-manager: Add subscribehandler Run() tests covering the sentinel topic declare, the no-sentinel-target case, and the topic-create and subscribe failure paths - bin-call-manager: Document the event subscriptions and the sentinel declare-before-bind behavior in docs/architecture.md - bin-timeline-manager: Same guard in its Run() loop, comparing against commonoutline.QueueNameSentinelEvent directly since its subscribeTargets is []commonoutline.QueueName - bin-timeline-manager: Add tests asserting the sentinel topic is declared before the sentinel subscribe and that a failed declare is returned; add the TopicCreate expectation to the existing bind-ordering regression test - bin-timeline-manager: Document the sentinel declare-before-bind behavior and the pre-existing boot-order sensitivity of the other targets in docs/architecture.md
transcribe-manager exited at boot whenever neither the GCP nor the AWS STT
client could be initialized: NewStreamingHandler returned a bare nil interface
and cmd/transcribe-manager turned that into a fatal error. That is a
disproportionate response, since every non-streaming capability of the service
works fine without an STT provider, and it is easy to hit accidentally (for
example a placeholder GCP key file and no AWS credentials). The constructor now
returns a disabled StreamingHandler instead, so the service boots and only the
streaming transcribe path reports the failure, per request.
- bin-transcribe-manager: Add pkg/streaminghandler/disabled.go with
NewDisabledStreamingHandler and the exported ErrSTTNotConfigured sentinel
("STT_NOT_CONFIGURED: no STT provider available"); Run() is a no-op so startup
completes, Start()/Stop() return ErrSTTNotConfigured
- bin-transcribe-manager: Return NewDisabledStreamingHandler() from
NewStreamingHandler when both providers are unavailable, instead of nil
- bin-transcribe-manager: Remove the now-unreachable streamingHandler == nil
fatal check from cmd/transcribe-manager/main.go
- bin-transcribe-manager: Add Test_NewDisabledStreamingHandler covering the
no-op Run() and the ErrSTTNotConfigured Start()/Stop() contract; relax
Test_NewStreamingHandler_NoProviders to the non-nil constructor contract, which
no longer depends on whether ambient GCP credentials happen to exist
- bin-transcribe-manager: Replace the "at least one provider must be configured
at startup" invariant in CLAUDE.md and docs/operations.md with the
degrade-instead-of-crash behavior, and add an STT_NOT_CONFIGURED row to the
failure-modes table
- bin-transcribe-manager: Remove the stale streamingHandler == nil fatal check left in transcribe-control (identical to the one already removed from transcribe-manager's main.go) - the constructor can no longer return nil - bin-transcribe-manager: Replace the disabled streaming handler's plain errors.New sentinel with a structured cerrors.Unavailable VoipbinError (reason STT_NOT_CONFIGURED), matching this codebase's existing error convention so the API's errorResponse() translator returns a typed error instead of an opaque 500
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix two boot-crash bugs found while investigating why a customer's VoIPBin sandbox install failed: bin-call-manager and bin-timeline-manager unconditionally subscribe to a RabbitMQ exchange owned by bin-sentinel-manager and crash-loop when it's not deployed (true for any non-Kubernetes deployment, including sandbox), and bin-transcribe-manager crashes at boot when no STT provider credentials are configured. Both are now non-fatal. Full investigation and design in the sandbox repo: docs/plans/2026-07-30-sandbox-install-reliability-design.md.