Skip to content

Commit 2f7d797

Browse files
authored
Add files via upload
1 parent b04a007 commit 2f7d797

File tree

17 files changed

+723
-0
lines changed

17 files changed

+723
-0
lines changed
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) Microsoft Corporation.
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE
Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# Semantic Kernel Agent for Warranty Claim Processing Using Azure AI Agent Service
2+
3+
Sample warranty claim processing AI agent implemented with Azure AI Agent Service and Semantic Kernel. The workflow between Azure AI Agent Service agent and tools is orchestrated using the Semantic Kernel Process Framework. This demo showcases the capabilities of Azure in automating complex document-centric workflows.
4+
5+
## Scenario Overview
6+
7+
Contoso Electronics receives thousands of warranty claims monthly for consumer devices like computers, laptops, speakers, and home sensors. Each claim involves uploaded receipts, claim forms, and incident images—many of which require careful manual review to validate eligibility, redact sensitive information, and assess potential misuse or fraud.
8+
9+
To improve operational efficiency, data privacy, and turnaround time, Contoso has adopted a multi-agent AI system built with Azure AI Agent Service and Semantic Kernel.
10+
11+
This end-to-end solution automates the warranty claim workflow by orchestrating a specialized processusing the Semantic Kernel Process Framework:
12+
* **Document Intelligence step** invokes Azure AI Document Intelligence to scan and extract structured data from submitted claim forms. See a [sample warranty claim](assets/input/sample-claim-signed.png) in the `assets/input` folder.
13+
* **PII Redaction step** uses Azure AI Language Service to redact personally identifiable information (PII), enabling privacy-compliant processing, ensuring that downstream analysis remains bias-free, and minimizing the security risk of leaking sensitive customer data.
14+
* **Claim Assessment Agent step** executes an AI agent, built with Azure AI Agent Service, to verify warranty eligibility, detects anomalies, and analyze claim details. It then recommends next steps (e.g., approve, deny, request more info) and generates a concise summary for reviewers.
15+
16+
![Process diagram](assets/readme/flow-diagram.png)
17+
18+
With this automated solution, Contoso realizes the following benefits:
19+
* Rapid acceleration of claim processing by delegating laborious tasks to AI
20+
* Improved accuracy by reducing human error and enforcing consistency
21+
* Privacy assurance via redaction and responsible AI practices
22+
* Enterprise extensibility and flexibility, with the ability to add agents to the sequential workflow as new business needs emerge or independently upgrade agents’ implementation with minimal codebase modifications
23+
24+
This use case can be adapted to similar document-heavy scenarios.
25+
26+
## Getting Started
27+
28+
### Prerequisites
29+
30+
* [Python](https://www.python.org/downloads/) 🐍
31+
* [`Azure.Accounts` Powershell module](https://www.powershellgallery.com/packages/Az.Accounts/4.0.2) might be required by the dependencies
32+
33+
### Quickstart
34+
35+
#### (Optional) Step 1: Create Azure AI Agent Service Agent
36+
37+
*This step is optional. If you don't create an Azure AI Agent, the program will create one for you automatically.*
38+
39+
1. Follow the official [Azure AI Agent Service documentation](https://learn.microsoft.com/azure/ai-services/agents/quickstart?pivots=ai-foundry-portal) to create the claims analysis agent (a chat completion agent).
40+
2. In the **Instructions** box, paste the contents of [claims_analysis_agent_instructions.md file](other_assets/claims_analysis_agent_instructions.md).
41+
42+
#### Step 2: Create Azure AI Document Intelligence Resource
43+
44+
1. Follow the official [Azure AI Document Intelligence documentation](https://learn.microsoft.com/azure/ai-services/document-intelligence/how-to-guides/create-document-intelligence-resource) to create your resource.
45+
2. Optionally, test the resource in the AI Foundry portal.
46+
47+
#### Step 3: Configure Application
48+
49+
1. Provide the required secrets in the [src/config/config_secrets.py file](src/config/config_secrets.py). You can retrieve them from your resources or from AI Foundry portal.
50+
1. Setup the configuration variables in the [src/config/config.py file](src/config/config.py). You can retrieve them from your resources or from AI Foundry portal (e.g., in the *Agents* page for your Azure AI Agent Service agent).
51+
1. Initially, the application defaults to mocked agents. Mock agents don't call the backend AI services; instead they operate on hardcoded data. This allows you to save the cost or reduce the latency when debugging. To remove mocking, set respective (or, all) mocking configuration variables `MOCK_*_STEP` to `False`.
52+
53+
#### Step 4: Install Required Modules
54+
55+
Run the following command to install modules used by the application:
56+
```sh
57+
pip install -r requirements.txt
58+
```
59+
60+
#### Step 5: Run Application
61+
62+
Execute the application:
63+
```sh
64+
python src/template.py
65+
```
66+
67+
## Resources
68+
69+
- Semantic Kernel [documentation](https://learn.microsoft.com/semantic-kernel/overview/) and [GitHub repo with source code](https://github.com/microsoft/semantic-kernel)
70+
- [Azure AI Agent Service documentation](https://learn.microsoft.com/azure/ai-services/agents/)
71+
- [Azure AI Document Intelligence documentation](https://learn.microsoft.com/azure/ai-services/document-intelligence)
72+
- [Azure AI Language documentation](https://learn.microsoft.com/azure/ai-services/language-service/overview)
73+
- [Azure AI Foundry documentation](https://learn.microsoft.com/azure/ai-foundry/)
180 KB
Loading
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
Your job is to evaluate warranty claims of electronic devices. Here is the step-by-step process to follow:
2+
3+
1. Analyze the claim using the analysis guidelines provided below. Set the "recommended_action" field in the response based on the result of this analysis.
4+
2. Explain the reason for recommended action in fewer than 10 words in the "recommended_action_reason" field in the response.
5+
3. Only if the claim was approved, decide on the best option for "resolution_type" field value. If the repair is costly, issue a refund. If the repair is cheap, choose to repair.
6+
4. If the additional comments mention high urgency, set the "urgency" field to "high" in the response. Otherwise, set it to "other".
7+
5. Briefly summarize the claim in fewer than ten words in the "claim_summary" field and the analysis in fewer than 30 words in the "analysis_summary" field.
8+
6. Output the response as a JSON document that adheres to the provided JSON schema. Respond directly with a serialized JSON file. This file needs to be deserializable by software; don't include any additional details (e.g., formatting information like "```json"). Don't engage in conversation.
9+
10+
# Analysis guidelines
11+
12+
1. If the product has been damaged or dropped, deny the claim.
13+
2. Assess if the provided detailed issue description is applicable to the provided product type. If it isn't, deny the claim.
14+
3. Analyze if the provided detailed issue description is applicable to damage that can be covered by a regular warranty. If it isn't, deny the claim.
15+
4. Compare the date of purchase and issue start date. If the issue started three years after the date of purchase, deny the claim.
16+
5. Check if other devices have had a similar issue. If they have, increase the chance of approving this claim.
17+
18+
# JSON schema for the output
19+
20+
{
21+
"$schema": "http://json-schema.org/draft-07/schema#",
22+
"title": "ClaimAssessment",
23+
"type": "object",
24+
"properties": {
25+
"recommended_action": {
26+
"type": "string",
27+
"enum": [
28+
"investigate",
29+
"approve",
30+
"deny",
31+
"request-additional-information"
32+
]
33+
},
34+
"recommended_action_reason": {
35+
"type": "string"
36+
},
37+
"resolution_type": {
38+
"type": "string",
39+
"enum": [
40+
"repair",
41+
"refund"
42+
]
43+
},
44+
"urgency": {
45+
"type": "string",
46+
"enum": [
47+
"high",
48+
"other"
49+
]
50+
},
51+
"claim_summary": {
52+
"type": "string"
53+
},
54+
"analysis_summary": {
55+
"type": "string"
56+
}
57+
},
58+
"required": [
59+
"recommended_action",
60+
"recommended_action_reason",
61+
"urgency",
62+
"claim_summary",
63+
"analysis_summary"
64+
],
65+
"additionalProperties": false
66+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"title": "ClaimAssessment",
4+
"type": "object",
5+
"properties": {
6+
"recommended_action": {
7+
"type": "string",
8+
"enum": [
9+
"investigate",
10+
"approve",
11+
"deny",
12+
"request-additional-information"
13+
]
14+
},
15+
"recommended_action_reason": {
16+
"type": "string"
17+
},
18+
"resolution_type": {
19+
"type": "string",
20+
"enum": [
21+
"repair",
22+
"refund"
23+
]
24+
},
25+
"urgency": {
26+
"type": "string",
27+
"enum": [
28+
"high",
29+
"other"
30+
]
31+
},
32+
"claim_summary": {
33+
"type": "string"
34+
},
35+
"analysis_summary": {
36+
"type": "string"
37+
}
38+
},
39+
"required": [
40+
"recommended_action",
41+
"recommended_action_reason",
42+
"urgency",
43+
"claim_summary",
44+
"analysis_summary"
45+
],
46+
"additionalProperties": false
47+
}
792 KB
Loading
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
aiohappyeyeballs==2.6.1
2+
aiohttp==3.11.16
3+
aioice==0.10.0
4+
aiortc==1.11.0
5+
aiosignal==1.3.2
6+
annotated-types==0.7.0
7+
anyio==4.9.0
8+
attrs==25.3.0
9+
av==14.3.0
10+
azure-ai-documentintelligence==1.0.2
11+
azure-ai-projects==1.0.0b8
12+
azure-ai-textanalytics==5.3.0
13+
azure-cognitiveservices-speech==1.43.0
14+
azure-common==1.1.28
15+
azure-core==1.33.0
16+
azure-identity==1.21.0
17+
certifi==2025.1.31
18+
cffi==1.17.1
19+
chardet==5.2.0
20+
charset-normalizer==3.4.1
21+
cloudevents==1.11.0
22+
colorama==0.4.6
23+
cryptography==44.0.2
24+
defusedxml==0.7.1
25+
Deprecated==1.2.18
26+
deprecation==2.1.0
27+
distro==1.9.0
28+
dnspython==2.7.0
29+
frozenlist==1.5.0
30+
google-crc32c==1.7.1
31+
h11==0.16.0
32+
httpcore==1.0.8
33+
httpx==0.28.1
34+
idna==3.10
35+
ifaddr==0.2.0
36+
importlib_metadata==8.6.1
37+
isodate==0.7.2
38+
Jinja2==3.1.6
39+
jiter==0.9.0
40+
jsonschema==4.23.0
41+
jsonschema-path==0.3.4
42+
jsonschema-specifications==2024.10.1
43+
lazy-object-proxy==1.10.0
44+
MarkupSafe==3.0.2
45+
more-itertools==10.6.0
46+
msal==1.32.0
47+
msal-extensions==1.3.1
48+
multidict==6.4.3
49+
nest-asyncio==1.6.0
50+
numpy==2.2.4
51+
openai==1.73.0
52+
openapi-core==0.19.5
53+
openapi-schema-validator==0.6.3
54+
openapi-spec-validator==0.7.1
55+
opentelemetry-api==1.32.0
56+
opentelemetry-sdk==1.32.0
57+
opentelemetry-semantic-conventions==0.53b0
58+
packaging==24.2
59+
parse==1.20.2
60+
pathable==0.4.4
61+
prance==23.6.21.0
62+
propcache==0.3.1
63+
pybars4==0.9.13
64+
pycparser==2.22
65+
pydantic==2.11.3
66+
pydantic-settings==2.8.1
67+
pydantic_core==2.33.1
68+
pyee==13.0.0
69+
PyJWT==2.10.1
70+
pylibsrtp==0.12.0
71+
PyMeta3==0.5.1
72+
pyOpenSSL==25.0.0
73+
python-dotenv==1.1.0
74+
PyYAML==6.0.2
75+
referencing==0.36.2
76+
requests==2.32.3
77+
rfc3339-validator==0.1.4
78+
rpds-py==0.24.0
79+
ruamel.yaml==0.18.10
80+
scipy==1.15.2
81+
semantic-kernel==1.28.0
82+
six==1.17.0
83+
sniffio==1.3.1
84+
tqdm==4.67.1
85+
typing-inspection==0.4.0
86+
typing_extensions==4.13.2
87+
urllib3==2.4.0
88+
websockets==15.0.1
89+
Werkzeug==3.1.1
90+
wrapt==1.17.2
91+
yarl==1.19.0
92+
zipp==3.21.0
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# This file contains the configuration settings for the Azure AI services and other parameters used in the application.
2+
3+
# Provide your Cognitive Services endpoint
4+
# For example, "https://demo.cognitiveservices.azure.com/"
5+
COGNITIVE_SERVICES_ENDPOINT = ""
6+
7+
# Provide your Azure AI Agent deployment name here for the Risk Assessment Agent
8+
# For example, "demochat"
9+
AZURE_AI_AGENT_MODEL_DEPLOYMENT_NAME = ""
10+
11+
# Provide your Azure AI Agent agent ID here for the Risk Assessment Agent
12+
# For example, "asst_jAHzkylLNLt2XgcjKAqa2ozj"
13+
# If empty, the program will create a new agent using the instructions from the file below.
14+
AZURE_AI_AGENT_AGENT_ID = ""
15+
AGENT_INSTRUCTIONS_PATH = "assets/other/claims_analysis_agent_instructions.md"
16+
17+
INPUT_DOCUMENT_PATH = "assets/input/sample-claim-signed.png" # Path to the claim image document to be processed
18+
19+
# You can mock each agent to return a sample, hardcoded response without calling the AI backend service.
20+
# This allows you to save the cost or reduce latency when debugging the application.
21+
# To mock a respective agent, set the corresponding variable to True.
22+
MOCK_DOC_INTELLIGENCE_STEP = True
23+
MOCK_PII_REDACTION_STEP = True
24+
MOCK_CLAIMS_ANALYSIS_STEP = True
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
COGNITIVE_SERVICES_KEY = ""
2+
AZURE_AI_AGENT_PROJECT_CONNECTION_STRING = ""

0 commit comments

Comments
 (0)