Microsoft Foundry
Python + AI Weekly Office Hours: Recordings & Resources #280
Replies: 175 comments 3 replies
-
|
2026/01/06: Do you think companies will create internal MCP servers for AI apps to connect to? Yes, this is already happening quite a bit. Common use cases include:
A particularly valuable use case is data science/engineering teams creating MCP servers that enable less technical folks (marketing, PMs, bizdev) to pull data safely without needing to write SQL. The pattern often starts with an engineer building an MCP server for themselves, sharing it with colleagues, adding features based on their needs, and growing from there. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/06: How do you set up Entra OBO (On-Behalf-Of) flow for Python MCP servers? 📹 5:48 The demo showed how to use the Graph API with the OBO flow to find out the groups of a signed-in user and use that to decide whether to allow access to a particular tool. The flow works as follows:
For the authentication dance, FastMCP handles the DCR (Dynamic Client Registration) flow since Entra itself doesn't support DCR natively. To test from scratch:
Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/06: Which MCP inspector should I use for testing servers with Entra authentication? 📹 20:24 The standard MCP Inspector doesn't work well with Entra authentication because it doesn't do the DCR (Dynamic Client Registration) dance properly. MCP Jam is recommended instead because it properly handles the OAuth flow with DCR. To set it up:
MCP Jam also has nice features like:
One note: enum values in tools don't yet show as dropdowns in MCP Jam (issue to be filed). Links shared: What's the difference between MCP Jam and LM Studio? 📹 34:19 LM Studio is primarily for playing around with LLMs locally. MCP Jam has some overlap since it includes a chat interface with access to models, but its main purpose is to help you develop MCP servers and apps. It's focused on the development workflow rather than just chatting with models. |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/06: How do you track LLM usage tokens and costs? 📹 28:04 For basic tracking, Azure portal shows metrics for token usage in your OpenAI accounts. You can see input tokens and output tokens in the metrics section. You can also:
If you use multiple providers, you need a way to consolidate the tracking. OpenTelemetry metrics could work but you'd need a way to hook into each system. |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/06: How do you keep yourself updated with all the new changes related to AI? 📹 30:32 Several sources recommended:
Particularly recommended:
Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/06: How do you build a Microsoft Copilot agent in Python with custom API calls? 📹 36:30 For building agents that work with Microsoft 365 Copilot (which appears in Windows Copilot and other Microsoft surfaces):
The agent framework team is responsive if there are issues. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/06: As a backend developer with a non-CS background, how do I learn about AI from scratch? 📹 46:39 Recommended approach:
Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/06: What's new with the RAG demo (azure-search-openai-demo) after the SharePoint data source was added? 📹 49:50 The main work is around improving ACL (Access Control List) support. The cloud ingestion feature was added recently, but it doesn't yet support ACLs. The team is working on making ACLs compatible with all features including:
A future feature idea: adding an MCP server to the RAG repo for internal documentation use cases, leveraging the Entra OBO flow for access control. |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/06: Do you think companies will create internal MCP servers for AI apps to connect to? 📹 53:53 Yes, this is already happening quite a bit. Common use cases include:
A particularly valuable use case is data science/engineering teams creating MCP servers that enable less technical folks (marketing, PMs, bizdev) to pull data safely without needing to write SQL. The pattern often starts with an engineer building an MCP server for themselves, sharing it with colleagues, adding features based on their needs, and growing from there. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/13: What advantages do other formats have over .txt for prompts? How do you improve prompts with DSPy and evals? 📹 4:55 Prompty is a template format that mixes Jinja and YAML together. The YAML goes at the top for metadata, and the rest is Jinja templating. Jinja is the most common templating system for Python (used by Flask, etc.). The nice thing about Jinja is you can pass in template variables—useful for customization, passing in citations, etc. Prompty turns the file into a Python list of chat messages with roles and contents. However, we're moving from Prompty to plain Jinja files because:
Recommendation: Keep prompts separate from code when possible, especially long system prompts. Use plain .txt or .md if you don't need variables, or Jinja if you want to render variables. With agents and tools, some LLM-facing text (like tool descriptions in docstrings) will inevitably live in your code—that's fine. For iterating on prompts: Run evaluations, change the prompt, and see whether it improves things. There are tools like DSPy and Agent Framework's Lightning that do automated prompt optimization/fine-tuning. Lightning says it "fine-tunes agents" but may actually be doing prompt changes. Most of the time, prompt changes don't make a huge difference, but sometimes they might. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/13: What is the future of AI and which specialization should I pursue? 📹 11:54 If you enjoy software engineering and full-stack engineering, it's more about understanding the models so you understand why they do what they do, but it's really about how you're building on top of those models. There's lots of interesting stuff to learn, and it really depends on you and what you're most interested in doing. |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/13: Which livestream series should I follow to build a project using several tools and agents, and should I use a framework? 📹 13:33 Everyone should understand tool calling before moving on to agents. From the original 9-part Python + AI series, start with tool calling, then watch the high-level agents overview. The upcoming six-part series in February will dive deeper into each topic, especially how to use Agent Framework. At the bare minimum, you should understand LLMs, tool calling, and agents. Then you can decide whether to do everything with just tool calling (you can do it yourself with an LLM that has tool calling) or use an agent framework like LangChain or Agent Framework if you think it has enough benefits for you. It's important to understand that agents are based on tool calling—it's the foundation of agents. The success and failure of agents has to do with the ability of LLMs to use tool calling. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/13: How does Azure manage the context window? How do I maintain a long conversation with a small context window? 📹 15:21 There are three general approaches:
With today's large context windows (128K, 256K), it's often easier to just wait for an error and tell the user to start a new chat, or do summarization when the error occurs. This approach is most likely to work across models since every model should throw an error when you're over the context window. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/13: How do we deal with context rot and how do we summarize context using progressive disclosure techniques? 📹 19:17 Read through Kelly Hong's (Chroma researcher) blog post on context rot. The key point is that even with a 1 million token context window, you don't have uniform performance across that context window. She does various tests to see when performance starts getting worse, including tests on ambiguity, distractors, and implications. A general tip for coding agents with long-running tasks: use a main agent that breaks the task into subtasks and spawns sub-agents for each one, where each sub-agent has its own focused context. This is the approach used by the LangChain Deep Agents repo. You can also look at how different projects implement summarization. LangChain's summarization middleware is open source—you can see their summary prompt and approach. They do approximate token counting and trigger summarization when 80% of the context is reached. Links shared:
How do I deal with context issues when using the Foundry SDK with a single agent? 📹 25:03 If you're using the Foundry SDK with a single agent (hosted agent), you can implement something like middleware through hooks or events. Another approach is the LangChain Deep Agents pattern: implement sub-agents as tools where each tool has a limited context and reports back a summary of its results to the main agent. For the summarization approach with Foundry agents, you'd need to figure out what events, hooks, or middleware systems they have available. |
Beta Was this translation helpful? Give feedback.
-
|
2026/01/13: Have you seen or implemented anything related to AG-UI or A2UI? 📹 29:02 AG-UI (Agent User Interaction Protocol) is an open standard introduced by the CopilotKit team that standardizes how front-end applications communicate with AI agents. Both Pydantic AI and Microsoft Agent Framework have support for AG-UI—they provide adapters to convert messages to the AG-UI format. The advantage of standardization is that if people agree on a protocol between backend and frontend, it means you can build reusable front-end components that understand how to use that backend. Agent Framework also supports different UI event stream protocols, including Vercel AI (though Vercel is a competitor, so support may be limited). These are adapters—you can always adapt output into another format if needed, but it's nice when it's built in. A2UI is created by Google with Consortium CopilotKit and relates to A2A (Agent-to-Agent). A2UI appears to be newer with less support currently in Agent Framework, though A2A is supported. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/09: What's new with PostgreSQL on Azure? 📹 48:00 A few things worth checking out from the Build 2026 PostgreSQL roundup:
|
Beta Was this translation helpful? Give feedback.
-
|
2026/06/09: Announcements 📹 50:59
|
Beta Was this translation helpful? Give feedback.
-
|
2026/06/17: Any learning resources for model evaluation? 📹 7:18 Pamela discussed several resources for learning about model evaluation. The most recent coverage was in the Host your agents on Foundry live stream series, which included a session on hosted agents that covered evaluations using the Foundry portal. There were also sessions on evaluations at Microsoft Build 2026. She then demoed the new rubric evaluators feature in Foundry, which is based on ASSERT research. The idea is to make evaluation easier by auto-generating custom evaluation rubrics. You describe your agent (e.g., "this agent queries data from a company knowledge base and gives back answers, it must always give back citations"), and Foundry generates a set of weighted sub-evaluators — things like evidential grounding, citation coverage and placement, citation traceability, uncertainty and limitations, and general quality. You can then run these rubrics against an agent, model, or data set, either through the portal or programmatically from Python. She also shared her experience with evaluations in the azure-search-openai-demo, where she upgraded the default model to gpt-5.4-mini. Finding a model that was better across the board on every metric was hard, and reasoning models no longer obey the seed parameter, making evaluations much more variable. Her approach is to run evals multiple times and calculate averages and statistical deviations. For general guidance on evaluation, she recommended Hamel Husain's blog for best practices on using LLMs as judges. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/17: Where should I start learning about AI? 📹 26:43 For people new to generative AI, Pamela recommended starting with the Microsoft IQ Deep Dive with Python series, a nine-part series covering the fundamentals. She also shared her blog post on How I learn about generative AI for other approaches and resources. The team may run the beginner series again if there's enough interest. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/17: Is Scout (from Build) the same as OpenClaw? 📹 31:00 Scout is basically a Microsoft-specific version of OpenClaw. It's a desktop app with automations and has really good integrations with Microsoft 365. OpenClaw is the open-source, bare version — just the Node repo that you can deploy yourself (e.g., on Azure Container Apps). By default, OpenClaw runs in an isolated container and doesn't have access to anything until you explicitly set up access. Scout adds those Microsoft 365 integrations on top. Pamela also mentioned she deployed OpenClaw on Azure Container Apps and fixed a few issues with doing it on Mac, and encouraged people to try it out. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/17: Do the three IQs come with attack protection? 📹 32:43 This is a continuation of a previous discussion about AI security and guardrails. Pamela recapped the security mechanisms available across the Microsoft AI stack. Thanks to that question from the previous office hours, the docs for detecting content filtering in the Responses API were improved. For data protection, Purview lets you label data with sensitivity levels (e.g., highly confidential, public) so agents know how data is classified. Matt recently added Purview integration to a branch of the azure-search-openai-demo. Pamela also explored Foundry IQ's new governance and enterprise AI security capabilities, and Microsoft Agent 365 — a control plane for agents that wraps together Entra, Defender, Purview, and Intune. Agent 365 provides threat detection, including Defender integration with Work IQ MCP that can block unsafe tool invocations. The discussion also touched on whether there should be a code analysis tool specifically for agent security — something that would look at your code and recommend improvements before you even deploy. The existing GitHub Copilot security review skill does code analysis, but it's generic rather than agent-specific. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/17: How many tools should ideally be in an agent? Should I use sub agents if there are too many tools? 📹 45:10 There's no hard and fast rule. Every model sees tools differently — this Tool Schema Rendering Atlas shows the actual differences in how tools are sent to models. The best approach is to run evaluations and check:
Reasons you might want to split into sub agents:
There doesn't appear to be a hard limit on number of tools per model — the practical limits are how smart the model is and how large its context window is. Models also vary in their ability to correctly select and call tools, so evaluation is key. Links shared: Can tools have sub tools? 📹 50:27 Not exactly. If you're trying to reduce the context of tool definitions, there's a concept of deferred tool loading (used by GitHub Copilot, for example) where only top-level tool groups are loaded initially, and specific tools are discovered and loaded on demand. But ultimately each tool is separate. If a tool needs to call another tool, that's essentially the same thing as an agent — you'd structure it as a tool calling another agent inside of it. However, you don't always need to reach for tools. Sometimes a single LLM call with structured outputs inside a tool is sufficient — you only need tools when there are multiple possible actions that could happen. |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/17: Announcements GitHub Copilot App now generally available 📹 1:03 The GitHub Copilot App is now generally available for macOS, Windows, and Linux. New features include canvases (custom UIs like to-do boards or deployment dashboards), workflows/automations (cron jobs — Pamela uses them for auto-accepting LinkedIn invites and daily issue triage), and bring-your-own-model support. The Copilot SDK and CLI also work with any open-source model, not just OpenAI. SpaceX to acquire Cursor for $60B; Cursor launches Origin 📹 0:31 SpaceX announced plans to acquire Anysphere (Cursor) for $60 billion. At the Cursor Compile conference, Cursor also launched Origin — a code storage and git hosting platform designed for agents. Anthropic Messages API and model swapping 📹 16:44 Pamela discussed work to improve Anthropic model support in Foundry. The Anthropic Messages API has a nice feature for RAG: structured citation support where you specify documents that can be cited, and the model returns citations in a structured format (instead of hoping the model puts citations in square brackets). She shared a model swap workshop repo that demonstrates swapping between multiple models (GPT-5.5, Mistral Large 3, Gemini K2.6, DeepSeek V4 Flash, and Anthropic models) deployed on Foundry. Links shared: VS Code blog: Token efficiency improvements in Copilot Chat 📹 22:53 A new blog post covers how GitHub Copilot improves token efficiency through prompt caching. When you send a new message, the majority of the prompt is cached and cheaper to process. However, switching models mid-conversation loses the prompt cache entirely, increasing cost. Anthropic's API gives more control over caching with explicit cache breakpoints for tool definitions, system prompts, and conversation segments. Upcoming Microsoft IQ live stream series 📹 25:18 A new live stream series on Microsoft IQ is coming at the end of July, covering Foundry IQ, Work IQ, and Fabric IQ, with office hours after each session. |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/23: Announcements 📹 0:58 Copilot & VS Code
Agent Framework
Industry
What Pamela's been up to
|
Beta Was this translation helpful? Give feedback.
-
|
2026/06/23: Discussion: New agent-framework loop middleware 📹 6:01 The Agent Framework Python 1.9.0 release added an AgentLoopMiddleware for re-running agents until some condition is met. While agents already run until they reach an answer, that answer may not match your actual goal. The middleware offers three loop strategies:
The release also added changes to approval handling, since human approval can interrupt loops. Additionally, in 1.7.0 they added a HarnessAgent — a batteries-included preconfigured agent with call history persistence, compaction, to-do mode, memory, and skills (similar to LangChain deep agents or the Copilot SDK). Links shared: Do loops without proper specs lead to high token consumption and undesired features? 📹 11:06 Yes — Pamela agreed that using loops without proper specs can lead to high token consumption. She gave the example of the Agent Framework's new judge loop sample, where the agent prompt says "explain why the sky is blue" but the judge checks for "includes at least one good joke." That's going to loop for a long time because the agent never got asked for a joke. The key insight: your agent prompt should have as much information as your judge prompt. If your output consistently fails the LLM-as-judge check, that criteria probably belongs in the system prompt too. The LLM-as-judge should be a second check, not the primary source of requirements. One exception: you might intentionally keep your system prompt shorter to save tokens, relying on the judge to catch edge cases. But that means your assumptions could break when you switch models. Also consider prompt caching — a few extra sentences in the system prompt isn't a big deal if you can take advantage of caching. Are the loops open to modify the middleware to add agents, evaluation, etc? 📹 14:27 Yes. The loop middleware is an agent middleware (the highest-level middleware in Agent Framework), which intercepts the You can add additional middleware, or since it's all open source, you can copy the loop middleware code and modify it to suit your needs. Pamela emphasized that middleware is generally pretty modifiable — it's designed so you can build your own. She recommends taking a framework's middleware as a basis and modifying it rather than feeling limited to what's provided out of the box. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/23: How do users authenticate to multiple MCP servers in production systems? 📹 20:07 For general users (not developers) using MCP with agents like GitHub Copilot or Claude Desktop, the new enterprise managed authorization feature helps. Admins can provision MCP connectors for their whole organization through their identity provider (starting with Okta), so users get connector access automatically on first login without per-app OAuth. Without enterprise managed auth, users individually authenticate to each MCP server. The coding agent (VS Code, Claude, etc.) acts as an OAuth client — it remembers connections, handles refresh tokens, and stores credentials. Pamela demonstrated this with the Hugging Face MCP server, showing how VS Code remembers previous logins and handles re-authentication automatically. The advantage of enterprise managed authorization is better auditing and fewer login prompts when multiple MCP servers go through the same identity provider. Currently Okta is the first supported identity provider; Entra does not support it yet. Links shared:
Is the authorization linked with specific MCP tool permissions? Where would that be configured? 📹 26:57 At the enterprise level, GitHub Copilot Enterprise/Business lets you create an MCP registry and set an allow list policy to control which MCP servers developers can use. However, this is at the server level, not the tool level — Pamela didn't see a way to enforce per-tool restrictions at the enterprise level. For per-tool restrictions, there are a few options:
Links shared:
Can Foundry Toolbox be used for MCP tool filtering/proxying? 📹 33:40 Yes — Foundry Toolbox is a possible option for this because when you add an MCP server to a toolbox, you can allow only certain tools. Pamela demonstrated creating a toolbox in the Foundry portal and showed that when configuring an MCP server within it, you can specify which tools to allow. However, the Foundry Toolbox MCP server currently doesn't work well as a user-facing MCP server connected directly from VS Code, because it uses Microsoft Entra which doesn't yet fully support MCP OAuth (DCR/CIMD). It works well for programmatic Python agents (like those built with Microsoft Agent Framework), but not for the interactive VS Code MCP flow. Pamela shared a example Agent Framework code for connecting to a Foundry Toolbox. When you create a toolbox in Foundry, that code will be auto-generated for you. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/23: Discussion: Tool search is now a first-class model capability 📹 42:40 Following up from a previous week's discussion about deferred tool loading and large MCP toolsets, Pamela shared that both OpenAI and Anthropic now train their models with tool search as a first-class capability. The models have actually been trained to search for tools when they don't have what they need, not just the harnesses (like coding agents) implementing it. This is specifically designed with MCP in mind — the OpenAI docs even reference MCP directly, recommending deferred loading for MCP servers. VS Code implements this too: when you add a new MCP server, it asks whether to always include the tools or defer loading, with defer as the default. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/06/23: Have you seen any use cases for agents doing write actions in production systems? 📹 44:53 Yes, several patterns:
The key principle: be most cautious when agents update existing human-created documents or delete anything. Writing something new (PRs, reports) that awaits human review is lower risk. Always set up evaluations and tests. |
Beta Was this translation helpful? Give feedback.
-
|
For production write actions, the control I would add is a typed action envelope before the tool call, not just a generic "human in the loop" switch. The useful boundary is: what resource is being changed, what side effect class it belongs to, what argument shape is approved, and how long that approval is valid. A PR creation, calendar invite, refund, document update, and permission change should not all collapse into the same "write" bucket. A pattern that has worked well for agent ops is:
The practical reason is incident review. When an agent sends the wrong email, updates the wrong record, or loops on a refund workflow, logs alone usually show that a tool was called. The receipt should show why that exact action was allowed, what it was supposed to do, and whether the final call stayed within the approved boundary. Disclosure: I work on Armorer Labs. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for keeping this thread updated. It is useful to have the office hour questions and resources collected in one place. One topic I would love to see covered more deeply in a future session is usage and cost evaluation for agent workflows. For normal chat, token usage is easier to reason about. But for agents, the real cost often comes from multiple hidden layers:
So the most useful metric may not be cost per prompt, but cost per completed task or cost per successful workflow. It would be great to see recommended patterns for tracking this in Python AI apps, especially when using multiple tools, multiple models, MCP servers, and evals together. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Each week, we hold weekly office hours about all things Python + AI in the Foundry Discord.
Join the Discord here: http://aka.ms/aipython/oh
This thread will list the recordings of each office hours, and any other resources that come out of the OH sessions. The questions and answers are automatically posted (based on the transcript) as comments in this thread.
June 23, 2026
Topics covered:
June 17, 2026
Topics covered:
June 9, 2026
Topics covered:
May 26, 2026
Topics covered:
May 19, 2026
Topics covered:
May 12, 2026
Topics covered:
April 28, 2026
Topics covered:
April 20, 2026
Topics covered:
April 13, 2026
Topics covered:
April 7, 2026
Topics covered:
March 31, 2026
Topics covered:
March 24, 2026
Topics covered:
March 17, 2026
Topics covered:
February 17, 2026
Topics covered:
February 10, 2026
Topics covered:
February 3, 2026
Topics covered:
January 27, 2026
Topics covered:
January 20, 2026
Topics covered:
January 13, 2026
Topics covered:
January 6, 2025
Topics covered:
Beta Was this translation helpful? Give feedback.
All reactions