Skip to content

Commit 95be3db

Browse files
tekrajchhetritekrajchhetri
committed
config file using ollama.
1 parent c53ccbb commit 95be3db

1 file changed

Lines changed: 226 additions & 0 deletions

File tree

‎example/coding_open-ended_responses/config/config_ollama.yaml‎

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
# This config file uses the ollama.
2+
agent_config:
3+
extractor_agent:
4+
role: >
5+
Extraction and classification agent
6+
goal: >
7+
Your goal is to provide an overview of the responses, highlighting key themes, patterns, notable insights and group similar responses into **semantic categories** based on their meaning or morphological similarities.
8+
backstory: >
9+
You are a trained observer who analyzes open-ended responses and classifies items into categories.
10+
llm:
11+
model: ollama/deepseek-r1:14b
12+
base_url: http://localhost:11434
13+
14+
alignment_agent:
15+
role: >
16+
Alignment agent
17+
goal: >
18+
Perform concept alignment to the extracted structured information {extracted_structured_information} by extractor_agent and map each "assigned_category". and return structured JSON output.
19+
backstory: >
20+
You are a trained semantic expert who analyzes "assigned_category" and performs the concept mapping.
21+
llm:
22+
model: ollama/deepseek-r1:14b
23+
base_url: http://localhost:11434
24+
25+
judge_agent:
26+
role: >
27+
Conflict resolution and judgement agent
28+
goal: >
29+
Your goal is to judge the output provided by alignment agent and determine conflict or issue in alignment or classification and assign the judge score.
30+
backstory: >
31+
You are an AI assistant specialized in analyzing and comparing classification and concept alignment results from extractor and alignment agent. Your expertise includes identifying similarities and differences in categorization, ensuring accurate representation of agreement.
32+
llm:
33+
model: ollama/deepseek-r1:14b
34+
base_url: http://localhost:11434
35+
36+
humanfeedback_agent:
37+
role: >
38+
Human feedback processor Agent
39+
goal: >
40+
Evaluate the {judged_structured_information_with_human_feedback} and review if everything is correct as human domain expert.
41+
backstory: >
42+
You are an AI assistant that acts as a human expert. Your task is to review classification and concept alignment results and correct as human expert.
43+
llm:
44+
model: ollama/deepseek-r1:14b
45+
base_url: http://localhost:11434
46+
47+
# agent task config
48+
task_config:
49+
extraction_task:
50+
description: >
51+
Read the provided {literature} carefully. It contains the questions and the open ended text response from a survey.
52+
53+
Your task is to do the following: Please do it in two steps.
54+
First step:
55+
1. Identify the main themes or categories that emerge from the responses.
56+
2. For each category, provide a brief description.
57+
Second Step:
58+
1. For each response:
59+
a) Read the text, i.e., questions and open ended answers {literature} carefully.
60+
b) Assign it to one of the categories from the first step if the response clearly matches that category’s description.
61+
c) Provide a brief explanation for the assignment in a "reason" field.
62+
d) If the response does not fit any existing category, leave the "assigned_category" field empty and provide a reason. Do not create new categories.
63+
e) If a response is ambiguous between two categories, list both and briefly explain in a “reason” field.
64+
65+
Provide the output in JSON format, see expected output.
66+
expected_output: >
67+
output format: json
68+
Example output:
69+
{
70+
"responses": [
71+
{
72+
"response_id": 1,
73+
"response_text": "I use various data analysis techniques to interpret results.",
74+
"assigned_category": "Data Analysis Techniques",
75+
"reason": "The response discusses methods and techniques used in data analysis."
76+
},
77+
{
78+
"response_id": 2,
79+
"response_text": "The instructional use of data is crucial for effective teaching.",
80+
"assigned_category": "Instructional Use",
81+
"reason": "This response highlights the importance of data in teaching."
82+
}
83+
]
84+
}
85+
agent_id: extractor_agent
86+
87+
alignment_task:
88+
description: >
89+
Perform concept alignment on the extracted structured information by mapping each "assigned_category" to a matching concept (class) from a given ontology or schema.
90+
91+
Inputs: {extracted_structured_information}
92+
93+
Instructions:
94+
alignment instructions:
95+
- For each assigned_category in `extracted_structured_information`, find the most appropriate matching concept from an ontology or schema (e.g., NCIT:C159271, schema.org and other related ontologies).
96+
- Concept alignment means associating a term with a known ontological class or identifier.
97+
- Use a local knowledge source or indexed search system to retrieve candidate concepts (labels or IDs) where possible and mark in remarks if a local knowledge source is used.
98+
- Record the matched ontology information for each term:
99+
- ontology_label - label from the ontology or schema class
100+
- assigned_category_mapped_to_ontology - iri of the ontology or schema class
101+
expected_output: >
102+
output format: json
103+
Example output:
104+
{
105+
"responses": [
106+
{
107+
"response_id": 1,
108+
"response_text": "I use various data analysis techniques to interpret results.",
109+
"assigned_category": "Data Analysis Techniques",
110+
"assigned_category_mapped_to_ontology": "EDAM:3365",
111+
"ontology_label": "Data architecture, analysis and design",
112+
"reason": "The response discusses methods and techniques used in data analysis."
113+
},
114+
{
115+
"response_id": 2,
116+
"response_text": "The instructional use of data is crucial for effective teaching.",
117+
"assigned_category": "Instructional Use",
118+
"assigned_category_mapped_to_ontology": "LOM:educational",
119+
"ontology_label": "educational use",
120+
"reason": "This response highlights the importance of data in teaching."
121+
}
122+
]
123+
}
124+
agent_id: alignment_agent
125+
126+
judge_task:
127+
description: >
128+
Evaluate the structured output from the alignment agent ({aligned_structured_information}). Assess both the quality of the assigned_category classification and its alignment with the ontology or schema. Assign a `judge_score` between 0 and 1, and append remarks justifying the evaluation.
129+
130+
Instructions:
131+
- Analyze the content of {aligned_structured_information} and evaluate how well the assigned_category is classified and aligned with the target ontology or schema.
132+
- Assign a `judge_score` between 0 and 1:
133+
- 1.0 = perfect alignment
134+
- 0.0 = misaligned or invalid
135+
- Intermediate values indicate partial alignment
136+
- If necessary adjust the "assigned_category" for "response_text" as well as the "assigned_category_mapped_to_ontology".
137+
- If user-modified fields (e.g., extracted terms) are present, do not overwrite them. Note their presence and evaluation in the remarks.
138+
- Ensure the final structure is schema-compliant and retains all original metadata and history.
139+
expected_output: >
140+
output format: json
141+
Example output:
142+
{
143+
"responses": [
144+
{
145+
"response_id": 1,
146+
"response_text": "I use various data analysis techniques to interpret results.",
147+
"assigned_category": "Data Analysis Techniques",
148+
"assigned_category_mapped_to_ontology": "EDAM:3365",
149+
"ontology_label": "Data architecture, analysis and design",
150+
"judge_score": 0.8,
151+
"reason": "The response discusses methods and techniques used in data analysis."
152+
},
153+
{
154+
"response_id": 2,
155+
"response_text": "The instructional use of data is crucial for effective teaching.",
156+
"assigned_category": "Instructional Use",
157+
"assigned_category_mapped_to_ontology": "LOM:educational",
158+
"ontology_label": "educational use",
159+
"judge_score": 0.9,
160+
"reason": "This response highlights the importance of data in teaching."
161+
}
162+
]
163+
}
164+
agent_id: judge_agent
165+
166+
humanfeedback_task:
167+
description: >
168+
Input: {judged_structured_information_with_human_feedback}
169+
170+
Objective: Evaluate and, if necessary, revise the output to ensure accurate alignment with the target ontology/schema based on feedback.
171+
172+
modification_context: {modification_context}
173+
user_feedback_text: {user_feedback_text}
174+
175+
Instructions:
176+
- If the input includes modifications made based on human/user feedback:
177+
- Detect and respect these changes (e.g., altered extracted terms).
178+
- Do not overwrite user-modified terms.
179+
- Instead, annotate in remarks that user-defined values were retained and evaluated accordingly.
180+
181+
Constraints:
182+
- Preserve all historical data.
183+
- Ensure outputs are traceable, auditable, and schema-compliant.
184+
expected_output: >
185+
output format: json
186+
Example output:
187+
{
188+
"responses": [
189+
{
190+
"response_id": 1,
191+
"response_text": "I use various data analysis techniques to interpret results.",
192+
"assigned_category": "Data Analysis Techniques",
193+
"assigned_category_mapped_to_ontology": "EDAM:3365",
194+
"ontology_label": "Data architecture, analysis and design",
195+
"judge_score": 0.8,
196+
"reason": "The response discusses methods and techniques used in data analysis."
197+
},
198+
{
199+
"response_id": 2,
200+
"response_text": "The instructional use of data is crucial for effective teaching.",
201+
"assigned_category": "Instructional Use",
202+
"assigned_category_mapped_to_ontology": "LOM:educational",
203+
"ontology_label": "educational use",
204+
"judge_score": 0.9,
205+
"reason": "This response highlights the importance of data in teaching."
206+
}
207+
]
208+
}
209+
agent_id: humanfeedback_agent
210+
211+
# embedding config
212+
embedder_config:
213+
provider: ollama
214+
config:
215+
api_base: http://localhost:11434
216+
model: nomic-embed-text:latest
217+
218+
# knowledge search config
219+
knowledge_config:
220+
search_key:
221+
- entity
222+
- label
223+
224+
# human in loop config
225+
human_in_loop_config:
226+
humanfeedback_agent: true

0 commit comments

Comments
 (0)