You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context: I saw the recent announcement regarding Multi-Turn Synthetic Goldens Generation which mentions support for "Turn-by-turn dialogue structure" and "Conversational Goldens instantly compatible with the Conversation Simulator."
The Issue: When I run synthesizer.generate_conversational_goldens_from_docs(), the resulting ConversationalGolden objects contain a Scenario and an Expected Outcome, but the turns attribute is empty (None).
I am trying to generate a dataset that looks like a literal script (User: Hi -> Bot: Hello), but I only seem to get the high-level scenario description.
My Code:
from deepeval.synthesizer import Synthesizer
context_config = ContextConstructionConfig(
chunk_size=1200,
chunk_overlap=300,
embedder=my_embedder,
critic_model=my_llm
)
evolution_config = EvolutionConfig(
num_evolutions=0
)
synthesizer = Synthesizer(model=my_llm,evolution_config=evolution_config)
print("🚀 Starting Generation...")
conversational_goldens = synthesizer.generate_conversational_goldens_from_docs(
document_paths=[file_path],
context_construction_config=context_config,
max_goldens_per_context=2
)
# When I check the output:
for golden in conversational_goldens:
print(f"Scenario: {golden.scenario}") # This works
print(f"Turns: {golden.turns}") # This returns None or is empty
Questions:
Does generate_conversational_goldens_from_docs generate the actual back-and-forth messages (turns) automatically, or is it only designed to generate the Scenario?
If it is only the Scenario, how do we achieve the "Turn-by-turn dialogue structure" mentioned in the announcement? Do we have to use the ConversationSimulator to generate the turns, or can the Synthesizer create synthetic turns directly from the docs?
Is there a specific parameter I am missing to tell the synthesizer to "write the actual dialogue"?
Any clarification or a small code example showing how to get a Golden object with populated turns would be greatly appreciated!
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Context: I saw the recent announcement regarding Multi-Turn Synthetic Goldens Generation which mentions support for "Turn-by-turn dialogue structure" and "Conversational Goldens instantly compatible with the Conversation Simulator."
The Issue: When I run synthesizer.generate_conversational_goldens_from_docs(), the resulting ConversationalGolden objects contain a Scenario and an Expected Outcome, but the turns attribute is empty (None).
I am trying to generate a dataset that looks like a literal script (User: Hi -> Bot: Hello), but I only seem to get the high-level scenario description.
My Code:
Questions:
Any clarification or a small code example showing how to get a Golden object with populated turns would be greatly appreciated!
Beta Was this translation helpful? Give feedback.
All reactions