Skip to content

Commit 0fcc833

Browse files
committed
Use configured answer strategy in eval report generator
It'd be useful to be able to set the `ANSWER_STRATEGY` env var when running the `evaluation:generate_report` Rake task to override the provider, as we allow in multiple other places in the app. Currently we build the question object manually which will use the database default `answer_strategy`. This commit explicitly sets it to be the Rails configuration value which uses the `ANSWER_STRATEGY` env var.
1 parent a589392 commit 0fcc833

2 files changed

Lines changed: 15 additions & 1 deletion

File tree

lib/evaluation/report_generator.rb

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,11 @@ def call(input_path)
2424
private
2525

2626
def build_question(question_message)
27-
Question.new(message: question_message, conversation: Conversation.new)
27+
Question.new(
28+
message: question_message,
29+
conversation: Conversation.new,
30+
answer_strategy: Rails.configuration.answer_strategy,
31+
)
2832
end
2933

3034
def absolute_govuk_url(path)

spec/lib/evaluation/report_generator_spec.rb

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,16 @@
7777
.to raise_error("File nonexistent.yml does not exist")
7878
end
7979

80+
it "uses the configured answer strategy" do
81+
allow(Rails.configuration).to receive(:answer_strategy).and_return("claude_structured_answer")
82+
83+
described_class.call(input_file.path)
84+
85+
expect(AnswerComposition::Composer).to have_received(:call).with(
86+
an_object_having_attributes(answer_strategy: "claude_structured_answer"),
87+
).twice
88+
end
89+
8090
it "returns the items" do
8191
items = described_class.call(input_file.path)
8292

0 commit comments

Comments
 (0)