Skip to content

feat: add FeedbackBuilder utility for structured reflective datasets#294

Open
rohitsandadi wants to merge 7 commits into
gepa-ai:mainfrom
rohitsandadi:feat/feedback-builder
Open

feat: add FeedbackBuilder utility for structured reflective datasets#294
rohitsandadi wants to merge 7 commits into
gepa-ai:mainfrom
rohitsandadi:feat/feedback-builder

Conversation

@rohitsandadi

Copy link
Copy Markdown
Contributor

Implements FeedbackBuilder (#264). Addresses #261 (anti-overfit), #262 (task drift), #288 (score visibility in reflective dataset).

Summary

FeedbackBuilder is a configurable utility in gepa.utils that replaces make_reflective_dataset() boilerplate. It replicates the existing side_info extraction logic, then layers on optional enrichments:

When no FeedbackBuilder is configured, the adapter falls back to existing behavior.

Changed files

  • src/gepa/utils/feedback_builder.py — new: FeedbackBuilder dataclass with build() method
  • src/gepa/utils/__init__.py — export FeedbackBuilder
  • src/gepa/adapters/optimize_anything_adapter/optimize_anything_adapter.py — accept + delegate to FeedbackBuilder
  • src/gepa/optimize_anything.py — add feedback_builder to ReflectionConfig, wire to adapter
  • tests/test_feedback_builder.py — 13 unit tests

Test plan

  • 13 tests covering base construction, each enrichment, combined features, import path, adapter integration
  • Full suite passes (383 tests)
  • ruff check + pyright clean

@semanticdiff-com

semanticdiff-com Bot commented Mar 26, 2026

Copy link
Copy Markdown

Review changes with  SemanticDiff

Changed Files
File Status
  src/gepa/adapters/optimize_anything_adapter/optimize_anything_adapter.py  5% smaller
  src/gepa/optimize_anything.py  0% smaller
  src/gepa/utils/__init__.py  0% smaller
  src/gepa/utils/feedback_builder.py  0% smaller
  tests/test_feedback_builder.py  0% smaller

@ECNUACRush

Copy link
Copy Markdown

Hi! I was looking into #288/#264 and noticed this PR already covers the default Score injection path and part of the FeedbackBuilder direction.

Since this PR has been idle for a while, would maintainers prefer to keep this PR as the main path, or would a smaller follow-up PR focused only on #288's default Score injection be useful?

I’m happy to help with a narrowly scoped PR or tests/docs if that would make this easier to review, and also happy to defer to this PR if it is still the preferred direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants