Skip to content

Chore: GenAI hygiene and Spring->GenAI read timeout - #355

Merged
DoPri merged 3 commits into
mainfrom
chore/genai-hygiene
Jul 18, 2026
Merged

Chore: GenAI hygiene and Spring->GenAI read timeout#355
DoPri merged 3 commits into
mainfrom
chore/genai-hygiene

Conversation

@DoPri

@DoPri DoPri commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Cleanup around the GenAI service plus one cross-service timeout fix. No behaviour change to the AI endpoints themselves.

  • Dockerfile cleanup. Dropped the three declared-but-unused build args (RELEASE_TYPE, APP_VERSION, IMAGE_TAG), the commented-out # ENTRYPOINT ["sleep", "600"] debug line, and the duplicate WORKDIR /app. Also pinned both base images by digest (ghcr.io/astral-sh/uv:python3.14-alpine and python:3.14-alpine), which is what the client and Spring Dockerfiles already do.
  • pyproject dedupe + version. dependencies listed uvicorn, pydantic, and langchain-openai twice; removed the duplicates. Bumped the version from 5.0.0 to 6.0.0 so genai lines up with the Spring services (also updated app/main.py's SERVICE_VERSION, which feeds the FastAPI version and the application_version metric, and the alexandria-genai entry in uv.lock).
  • Spring to GenAI read timeout. Both GenAiClients (knowledgebase- and qa-service) used a 30s read timeout, but the GenAI chat budget is LLM_TIMEOUT_SECONDS=60 with 2 retries, i.e. up to 180s worst case. So Spring could abandon a long-document request at 30s while GenAI kept working (and spending tokens) on something nobody was waiting for. Raised the read timeout to 200s so it sits above GenAI's worst case, rather than shortening GenAI, so the long-document requests the system is built for don't get cut off mid-flight.

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Docs
  • CI / infra

Definition of Done

  • CI is green
  • If the API changed: api/openapi.yaml is updated and lint passes
  • If a service was added or restructured: docker-compose still comes up cleanly (docker compose up)
  • Tests added or updated for the changed behaviour
  • Docs updated where it matters (README, ADRs, comments)

Notes for the reviewer

  • On the timeout: 200s covers the chat model's worst case (60s x 3). The qa /ask path does an embedding call plus the chat call, so in a pathological double-worst-case (both maxing out their retries) it could exceed 200s, but that only happens if GenAI is already badly degraded. If you'd rather cap the total instead, the alternative was shortening GenAI's own budget; I went with raising Spring per our discussion.

@DoPri
DoPri requested a review from ladu-tu as a code owner July 18, 2026 19:01
@DoPri DoPri added the genai GenAI service label Jul 18, 2026
@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@DoPri, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 57 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e40c3a94-a73e-4d44-b881-762a530187c2

📥 Commits

Reviewing files that changed from the base of the PR and between e76d670 and 43449f9.

⛔ Files ignored due to path filters (1)
  • services/genai/uv.lock is excluded by !**/*.lock
📒 Files selected for processing (5)
  • services/genai/Dockerfile
  • services/genai/app/main.py
  • services/genai/pyproject.toml
  • services/spring/knowledgebase-service/src/main/java/com/alexandria/knowledgebase/integration/GenAiClient.java
  • services/spring/qa-service/src/main/java/com/alexandria/qa/integration/GenAiClient.java
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/genai-hygiene

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ladu-tu ladu-tu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch regarding the Spring timeout!

@DoPri
DoPri merged commit e1a5693 into main Jul 18, 2026
34 checks passed
@DoPri
DoPri deleted the chore/genai-hygiene branch July 18, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

genai GenAI service

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants