Skip to content

Commit abd51bd

Browse files
ZyysurelyyingyingzhaozyyingakailaCopilotjohanste
authored
[feat] new azure-ai-agentserver-optimization package for load_config (#47096)
* [feat] new azure-ai-agentserver-optimization package for load_config * fix comments and guard path * more checks fix * depend on scaffolder * update warning * feat: add candidate_id parameter to load_config() for header-based resolution Enable single-version candidate resolution by accepting candidate_id as a parameter (e.g. from the X-Optimization-Candidate-Id request header) instead of requiring an env var per deployed version. The parameter takes precedence over the OPTIMIZATION_CANDIDATE_ID env var and is threaded through to both the resolver API and local directory lookup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: auto-extract candidate_id from X-Optimization-Candidate-Id header load_config() now automatically reads the candidate ID from the X-Optimization-Candidate-Id request header (Flask/Quart) when no explicit candidate_id parameter is passed. Resolution order: 1. Explicit candidate_id param 2. X-Optimization-Candidate-Id request header (auto-detected) 3. OPTIMIZATION_CANDIDATE_ID env var Also exports OPTIMIZATION_CANDIDATE_HEADER constant for agents that need to reference the header name. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * revert: remove candidate_id parameter and header auto-extract * address comments * fix tests * Refactor resolver error handling: fail-fast on API/skill errors, warn-only on persist - Move DefaultAzureCredential and sys imports to module top-level - Remove PyYAML ImportError fallback (formal dependency) - Make metadata.yaml parse errors raise ValueError - Extract _fetch_candidate_config() helper from resolve_candidate() - Raise ValueError on config fetch, manifest fetch, and skill download failures - Keep persist-to-disk failures as warnings (non-fatal) - Skip skill download when persist fails (try/except/else pattern) - Remove pydantic dependency from all test files (plain Python stubs) - Update tests to match fail-fast semantics (pytest.raises for ValueError) - Add tests for _fetch_candidate_config and download exception cases - Fix DefaultAzureCredential mock patch path in TestBuildClient * Fix mypy: add type: ignore[import-untyped] for yaml import * address comments * Fix docstring on CandidateConfig * fix pylint * [fix] multiple-line skill descirption issue * don't auto split --------- Co-authored-by: zyysurely <yingyingzhao@microsoft.com> Co-authored-by: Ashish Kaila <ashishkaila@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Johan Stenberg (MSFT) <johan.stenberg@microsoft.com>
1 parent ea4c378 commit abd51bd

21 files changed

Lines changed: 4182 additions & 0 deletions

‎sdk/agentserver/azure-ai-agentserver-optimization/CHANGELOG.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Release History
2+
3+
## 1.0.0b1 (2026-05-24)
4+
5+
### Features Added
6+
7+
- Initial beta release.
8+
- `load_config(*, config_dir)` — single-call config loader with 4-priority resolution and graceful fallback.
9+
- `load_skills_from_dir(path)` — load skills from a directory on demand (not loaded inline by `load_config`).
10+
- `OptimizationConfig` with instructions, model, temperature, skills, skills_dir, tool_definitions, source, and candidate_id.
11+
- `OptimizationConfig.apply_tool_descriptions(tools)` — patch `__doc__`, `.description`, and `input_model` parameter descriptions on @tool-decorated functions from optimized tool definitions.
12+
- `OptimizationConfig.compose_instructions()` — append skill catalog to instructions.
13+
- `CandidateConfig` — typed representation of the resolver API payload.
14+
- `Skill` — learned skill model (name, description, body).
15+
- 4-priority resolution order:
16+
1. Inline JSON via `OPTIMIZATION_CONFIG` env var.
17+
2. Resolver API via `OPTIMIZATION_CANDIDATE_ID` + `OPTIMIZATION_RESOLVE_ENDPOINT` (endpoint is the full job-scoped URL).
18+
3. Local directory layout (`OPTIMIZATION_LOCAL_DIR` or `config_dir` param, defaults to `.agent_configs/`).
19+
4. No config found → returns `None`.
20+
- Local directory layout: `metadata.yaml` + `instructions.md` + `tools.json` + `skills/` per candidate, with `baseline/` fallback.
21+
- Tool definitions use the OpenAI function-calling list format exclusively.
22+
- Skill loading from `SKILL.md` files with YAML frontmatter.
23+
- Resolver API persists fetched configs and skill files to local directory for offline use.
24+
- Path traversal (zip-slip) protection on skill file downloads from the API.

‎sdk/agentserver/azure-ai-agentserver-optimization/LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
Copyright (c) Microsoft Corporation.
2+
3+
MIT License
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎sdk/agentserver/azure-ai-agentserver-optimization/MANIFEST.in‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
include *.md
2+
include LICENSE
3+
recursive-include tests *.py
4+
recursive-include samples *.py *.md
5+
include azure/__init__.py
6+
include azure/ai/__init__.py
7+
include azure/ai/agentserver/__init__.py
8+
include azure/ai/agentserver/optimization/py.typed

‎sdk/agentserver/azure-ai-agentserver-optimization/README.md‎

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
# Azure AI Agent Server Optimization client library for Python
2+
3+
The `azure-ai-agentserver-optimization` package provides a drop-in config loader for optimization-ready Azure AI Hosted Agents. A single `load_config()` call resolves optimization parameters (instructions, model, temperature, skills, tool definitions) from multiple sources with graceful fallback — your agent works unchanged when not running under optimization.
4+
5+
## Getting started
6+
7+
### Install the package
8+
9+
```bash
10+
pip install azure-ai-agentserver-optimization
11+
```
12+
13+
### Prerequisites
14+
15+
- Python 3.10 or later
16+
17+
## Key concepts
18+
19+
### Resolution Order
20+
21+
`load_config()` resolves from four sources in order — first match wins:
22+
23+
| Priority | Source | Env vars required | Description |
24+
|----------|--------|-------------------|-------------|
25+
| 1 | **Inline JSON** | `OPTIMIZATION_CONFIG` | Full config as a JSON string. Used by temporary agent versions during evaluation. |
26+
| 2 | **Resolver API** | `OPTIMIZATION_CANDIDATE_ID`, `OPTIMIZATION_RESOLVE_ENDPOINT` | Fetches the candidate config from the remote optimization service and persists it to the local directory. The endpoint should be the full job-scoped URL. |
27+
| 3 | **Local directory** | `OPTIMIZATION_LOCAL_DIR` (optional, defaults to `.agent_configs/`) | Reads from `<local_dir>/<candidate_id>/` or `baseline/` as fallback. |
28+
| 4 | **No config** | *(none)* | Returns `None`. |
29+
30+
When `load_config()` encounters an invalid config source (e.g. malformed JSON env var, unreadable metadata), it raises `ValueError`. Other exceptions (e.g. network errors from the resolver API) propagate to the caller.
31+
32+
### Environment Variables
33+
34+
| Variable | Description |
35+
|----------|-------------|
36+
| `OPTIMIZATION_CONFIG` | Inline JSON config (Priority 1). |
37+
| `OPTIMIZATION_CANDIDATE_ID` | Candidate ID for resolver API or local folder lookup. |
38+
| `OPTIMIZATION_RESOLVE_ENDPOINT` | Full job-scoped URL of the optimization service (e.g. `https://host/api/projects/proj/agents_optimization_job/job-1`). |
39+
| `OPTIMIZATION_LOCAL_DIR` | Path to the local config directory (default: `.agent_configs/`). |
40+
41+
### Local Directory Layout
42+
43+
The local config directory **must** exist with at least a `baseline/` folder before the agent can start to do optimization. When running under optimization, the resolver API (Priority 2) persists candidate configs into the same layout. If a `<candidate_id>/` folder exists it takes precedence; otherwise `baseline/` is used as the fallback.
44+
45+
```
46+
.agent_configs/
47+
├── baseline/ # required — default candidate used at startup
48+
│ ├── metadata.yaml # model, temperature, file pointers
49+
│ ├── instructions.md # system prompt
50+
│ ├── tools.json # tool definitions (OpenAI function-calling list format)
51+
│ └── skills/ # learned skills
52+
│ └── <skill_name>/
53+
│ └── SKILL.md
54+
└── <candidate_id>/ # optional — created by resolver API, same layout as baseline/
55+
├── metadata.yaml
56+
├── instructions.md
57+
├── tools.json
58+
└── skills/
59+
└── <skill_name>/
60+
└── SKILL.md
61+
```
62+
63+
#### `metadata.yaml`
64+
65+
Points to the other files and sets model parameters. All fields are optional:
66+
67+
```yaml
68+
model: gpt-4o
69+
temperature: 0.7
70+
instruction_file: instructions.md
71+
skill_dir: skills
72+
tool_file: tools.json
73+
```
74+
75+
#### `instructions.md`
76+
77+
The system prompt for the agent — plain text or Markdown:
78+
79+
```markdown
80+
You are a travel assistant. Help users search flights, book hotels,
81+
and answer questions about company travel policy.
82+
```
83+
84+
#### `tools.json`
85+
86+
Tool definitions in the OpenAI function-calling list format:
87+
88+
```json
89+
[
90+
{
91+
"type": "function",
92+
"function": {
93+
"name": "lookup_policy",
94+
"description": "Look up the company travel policy.",
95+
"parameters": {
96+
"type": "object",
97+
"properties": {
98+
"dept": {"type": "string", "description": "Department name"}
99+
}
100+
}
101+
}
102+
}
103+
]
104+
```
105+
106+
#### `skills/*/SKILL.md`
107+
108+
Each skill lives in its own subfolder with a `SKILL.md` file. An optional YAML frontmatter block provides the name and description; the rest is the skill body:
109+
110+
```markdown
111+
---
112+
name: budget-checker
113+
description: Check whether a trip is within budget.
114+
---
115+
116+
Compare the trip cost against the department's remaining travel budget
117+
and return APPROVED or DENIED with a reason.
118+
```
119+
120+
### OptimizationConfig Properties
121+
122+
| Property | Type | Description |
123+
|----------|------|-------------|
124+
| `instructions` | `str \| None` | System prompt (optimized or default). |
125+
| `model` | `str \| None` | Model deployment name. |
126+
| `temperature` | `float \| None` | Sampling temperature. |
127+
| `skills` | `list[Skill]` | Learned skills (from inline config). |
128+
| `skills_dir` | `str \| None` | Path to skills directory (for on-demand loading). |
129+
| `tool_definitions` | `list[dict]` | Optimized tool definitions (OpenAI function-calling format). |
130+
| `source` | `str` | Where the config was loaded from. |
131+
| `candidate_id` | `str \| None` | Candidate ID (when resolved via API or local folder). |
132+
| `has_skills` | `bool` | Whether skills are available (inline or via skills_dir). |
133+
134+
### Public API
135+
136+
| Export | Type | Description |
137+
|--------|------|-------------|
138+
| `load_config(*, config_dir)` | function | Load optimization config with 4-priority resolution. |
139+
| `load_skills_from_dir(path)` | function | Load skills from a directory of `SKILL.md` files. |
140+
| `OptimizationConfig` | class | Resolved config with instructions, model, temperature, skills_dir, tool_definitions. |
141+
| `OptimizationConfig.apply_tool_descriptions(tools)` | method | Patch `__doc__`, `.description`, and parameter descriptions on tool functions. |
142+
| `OptimizationConfig.compose_instructions()` | method | Return instructions with skill catalog appended. |
143+
| `CandidateConfig` | class | Typed representation of the resolver API response. |
144+
| `Skill` | class | A learned skill (name, description, body). |
145+
146+
## Examples
147+
148+
### Basic usage
149+
150+
```python
151+
from azure.ai.agentserver.optimization import load_config
152+
153+
config = load_config() # uses .agent_configs/baseline/
154+
config = load_config(config_dir="my_configs") # custom directory
155+
156+
if config is None:
157+
print("No optimization config found")
158+
else:
159+
print(config.instructions) # optimized system prompt
160+
print(config.model) # optimized model name
161+
print(config.temperature) # optimized temperature
162+
print(config.tool_definitions) # optimized tool definitions (list)
163+
print(config.source) # "env:OPTIMIZATION_CONFIG", "api:candidate:abc", "local:...", etc.
164+
```
165+
166+
### Apply optimized tool descriptions
167+
168+
```python
169+
from azure.ai.agentserver.optimization import load_config
170+
171+
config = load_config()
172+
173+
# Your @tool-decorated functions
174+
def search_flights(origin: str, destination: str):
175+
"""Search for flights."""
176+
...
177+
178+
def book_hotel(city: str):
179+
"""Book a hotel room."""
180+
...
181+
182+
# Patches __doc__ and .description on matching tools with optimized descriptions
183+
config.apply_tool_descriptions([search_flights, book_hotel])
184+
```
185+
186+
### Load skills on demand
187+
188+
```python
189+
from pathlib import Path
190+
from azure.ai.agentserver.optimization import load_config, load_skills_from_dir
191+
192+
config = load_config()
193+
194+
# Skills are not loaded inline — load them when needed
195+
if config.skills_dir:
196+
skills = load_skills_from_dir(Path(config.skills_dir))
197+
for skill in skills:
198+
print(f"{skill.name}: {skill.description}")
199+
```
200+
201+
## Troubleshooting
202+
203+
Enable debug logging to see resolution details:
204+
205+
```python
206+
import logging
207+
logging.getLogger("azure.ai.agentserver.optimization").setLevel(logging.DEBUG)
208+
```
209+
210+
Common issues:
211+
- **Config not loading from resolver API** — ensure both env vars are set: `OPTIMIZATION_CANDIDATE_ID` and `OPTIMIZATION_RESOLVE_ENDPOINT`.
212+
- **Local directory not found** — check that `OPTIMIZATION_LOCAL_DIR` points to an existing directory, or ensure `.agent_configs/` exists relative to your main script.
213+
- **`load_config()` returns `None`** — no config source was found; set up a baseline folder or set the appropriate env vars.
214+
215+
## Next steps
216+
217+
- [Azure SDK for Python documentation](https://learn.microsoft.com/azure/developer/python/)
218+
- [Contributing guide](https://github.com/Azure/azure-sdk-for-python/blob/main/CONTRIBUTING.md)
219+
220+
## Contributing
221+
222+
This project welcomes contributions and suggestions. See [CONTRIBUTING.md](https://github.com/Azure/azure-sdk-for-python/blob/main/CONTRIBUTING.md) for details.

‎sdk/agentserver/azure-ai-agentserver-optimization/azure/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__path__ = __import__("pkgutil").extend_path(__path__, __name__)

‎sdk/agentserver/azure-ai-agentserver-optimization/azure/ai/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__path__ = __import__("pkgutil").extend_path(__path__, __name__)

‎sdk/agentserver/azure-ai-agentserver-optimization/azure/ai/agentserver/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__path__ = __import__("pkgutil").extend_path(__path__, __name__)

‎sdk/agentserver/azure-ai-agentserver-optimization/azure/ai/agentserver/optimization/__init__.py‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# ---------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# ---------------------------------------------------------
4+
5+
"""Agent Optimization — Config loader for optimization-ready hosted agents.
6+
7+
One import, one call::
8+
9+
from azure.ai.agentserver.optimization import load_config
10+
11+
config = load_config() # uses .agent_configs/baseline/
12+
config = load_config(config_dir="my_configs") # custom directory
13+
14+
Resolution order (first match wins):
15+
1. OPTIMIZATION_CONFIG env var → inline JSON (used by temp agent versions)
16+
2. OPTIMIZATION_CANDIDATE_ID + ENDPOINT → resolver API → full config + skills
17+
3. Local directory (config_dir or .agent_configs/)
18+
→ metadata.yaml + instructions.md + tools.json + skills/
19+
4. No config found → returns ``None``.
20+
"""
21+
22+
from azure.ai.agentserver.optimization._config import load_config, load_skills_from_dir
23+
from azure.ai.agentserver.optimization._models import (
24+
CandidateConfig,
25+
OptimizationConfig,
26+
Skill,
27+
)
28+
from azure.ai.agentserver.optimization._version import VERSION
29+
30+
__all__ = [
31+
"CandidateConfig",
32+
"OptimizationConfig",
33+
"Skill",
34+
"load_config",
35+
"load_skills_from_dir",
36+
]
37+
__version__ = VERSION

0 commit comments

Comments
 (0)