Foundry IQ Response API - 55 to 90 second delay

[codeputer]

Responses file_search Latency and Timeout

Date prepared: 2026-06-01
Updated: 2026-06-05 with post-connection-removal staging diagnostic evidence.

Support Ticket Summary

We need Azure support to investigate intermittent high-latency and timeout behaviour when using Azure AI Foundry / Azure OpenAI Responses API with the file_search tool against Foundry vector store IDs.

file_search is understood here as Foundry file-backed vector retrieval over a vector store. ContentTraker is not issuing a raw Azure AI Search/vector query directly; the application call boundary is Responses API create_response, and Foundry/Responses orchestrates the file-backed vector retrieval and model response.

This application is not using a Persistent Agent and is not using an Azure AI Foundry knowledge base for this path. The runtime call is a direct Responses API create_response call with:

• tools[0].type = "file_search"
• tools[0].vector_store_ids = [...]
• Azure AI Foundry project-scoped OpenAI v1 endpoint
• RBAC auth via managed identity / DefaultAzureCredential

The vector store is small, but file_search calls are taking 47 to 88+ seconds when they succeed and sometimes exceed our 90-second application timeout. A 2026-06-04 warm rerun still timed out, so this does not appear to be only first-call/cold-start behavior. A 2026-06-05 staging run after deleting a stale Foundry project Azure AI Search connection still executed the Responses file_search path, but KBA again hit the 90-second timeout.

Impact

The feature is user-facing Search Diagnostics and production retrieval. Slow file_search makes responses appear hung and forces us to fall back to local ContentTraker evidence after the 90-second budget.

Environment

• Azure subscription: redacted
• Azure AI resource group: ArticleMngResources_RG
• Azure AI resource: articlemngrprj-resource
• Azure AI resource kind: AIServices
• Region: eastus2
• Foundry project endpoint: redacted
• Responses endpoint used by code: https://articlemngrprj-resource.services.ai.azure.com/api/projects/articlemngrprj/openai/v1/responses
• Model deployment: gpt-5.4-pro (now changed to mini, which improved timings)
• Embedding deployment: text-embedding-3-large
• Web app environment: Azure Container Apps staging
• User web revision: userweb--0000123
• User web image: redacted
• Application build chip: v1.0.5 - f6e514a

Vector Stores Involved

Workspace: BackTheApp.software

• Workspace ID: 019d35a8-4eeb-7a1e-9d5f-040b08e60841
• KBA workspace vector store ID: vs_uLY7yQIK6ca1iwjqXk1ETgPU
• KPA project vector store ID: vs_qfEXzB7pNeYZlQfN4SqbwouB

The slow path is consistently the KBA workspace vector store. KPA returns no vector match in about 11 to 13.5 seconds.

Reproduction

1. Open staging: https://userweb.staging.contenttraker.com/
2. Sign in as a service admin.
3. Open Search Diagnostics.
4. Select workspace: BackTheApp.software., and the Project for Codex
5. Use question: what is the purpose of this workspace
6. Run diagnostics.

The page runs KBA and KPA file_search probes in parallel. Each probe has a 90-second budget. KUA is not run by this page.

Observed Results

Application log query over the staging Log Analytics workspace produced these file_search timings:

UTC time	Revision	Lane	Status	Elapsed ms	Timeout budget ms	Request ID
2026-06-01T18:06:27Z	userweb--0000121	KPA	NoMatchingEvidence	13413	90000	61f9d1d4-feb4-4f4f-8213-a863f6bb4c42
2026-06-01T18:07:01Z	userweb--0000121	KBA	Succeeded	47954	90000	c42101c2-e6f5-440f-9055-9520287723cf
2026-06-01T18:53:19Z	userweb--0000122	KPA	NoMatchingEvidence	11147	90000	85966375-6682-43b4-b5ad-433db458bfdc
2026-06-01T18:54:36Z	userweb--0000122	KBA	Succeeded	88438	90000	72445806-6d18-447f-91d4-857175ab305b
2026-06-01T19:10:28Z	userweb--0000123	KPA	NoMatchingEvidence	11149	90000	a9b25597-4d71-4da2-ae2c-217d7a3b4856
2026-06-01T19:11:47Z	userweb--0000123	KBA	TimedOut	90001	90000	none
2026-06-01T19:13:26Z	userweb--0000123	KPA	NoMatchingEvidence	12686	90000	65290065-d0fb-4e10-bc3d-456b85b522b4
2026-06-01T19:14:43Z	userweb--0000123	KBA	TimedOut	89999	90000	none
2026-06-01T19:26:31Z	userweb--0000123	KPA	NoMatchingEvidence	13495	90000	8ed18193-d710-4f5b-b010-5c77dd4297f9
2026-06-01T19:27:13Z	userweb--0000123	KBA	Succeeded	55846	90000	a87882fa-9ba2-4083-9270-57d9db000b24

2026-06-04 Search Diagnostics Provider-Stage Timing

Local Search Diagnostics was run from https://localhost:7333/search-diagnostics against the same workspace and question:

• Workspace: BackTheApp.software
• Workspace ID: 019d35a8-4eeb-7a1e-9d5f-040b08e60841
• Question: what is the purpose of this workspace
• Local corpus: 22 digital assets, 3 active projects, 21 conversation threads, 117 conversation turns
• KBA binding: Foundry vector store present
• KPA binding: no selected-project vector binding in this local run, so KPA used local ContentTraker fallback

First current run:

Component	Status	Elapsed ms	Client request ID	Azure request ID	Token ms	HTTP send to headers ms	Body read ms	JSON map ms
KBA file_search	VectorStoreSucceeded	71456	contenttraker-fc10d744a7e54615b1d0376ad1991d3b	ae474a7d-fab2-41b7-a484-0b6c799fba59	9360	62075	0	3
KPA local fallback	LocalOnly	51	n/a	n/a	n/a	n/a	n/a	n/a
Answer synthesis	Succeeded	98761	contenttraker-4ed30dd8cdf143e2a213a0f817a9a6cb	b11e4eaf-9438-4a7d-81f2-2f83a8315f8c	4430	94326	0	0

Warm rerun after the application was already started and the prior diagnostic had completed:

Component	Status	Elapsed ms	Client request ID	Azure request ID	Token ms	HTTP send to headers ms	Body read ms	JSON map ms
KBA file_search	VectorStoreTimedOutUsedLocalFallback	90007	not captured before current instrumentation fix	none	n/a	n/a	n/a	n/a
KPA local fallback	LocalOnly	17	n/a	n/a	n/a	n/a	n/a	n/a
Answer synthesis	timed out	104622	contenttraker-f86ff96bd9164239a8ebb1cd8cf2a860	none	4585	not returned before timeout	0	0

Post-fix rerun after restarting localhost with current correlation/timing instrumentation:

Component	Status	Elapsed ms	Client request ID	Azure request ID	Token ms	HTTP send to headers ms	Body read ms	JSON map ms
KBA file_search	VectorStoreSucceeded	88968	contenttraker-df9c8cba0283498d9071f07deed198c6	f19a3edc-a466-49e6-b318-fa3f5912082c	5295	83657	0	2
KPA local fallback	LocalOnly	53	n/a	n/a	n/a	n/a	n/a	n/a
Answer synthesis	Succeeded	100568	contenttraker-f197dd1bea574c1ba54c33c2862b8b3f	26992faf-12b1-4c88-a198-7be6feebf6e4	3622	96941	0	0

The warm rerun is important because it removes a simple cold-start explanation. The current successful reruns also show the app-side CPU/body/mapping work is negligible: the dominant interval is waiting for Responses/Foundry to return response headers. In the post-fix rerun, KBA succeeded only about one second before the 90-second application budget, and answer synthesis took about 100.6 seconds.

2026-06-05 Staging Run After Removing Stale Foundry AI Search Connection

Before this run, the stale Foundry project connection named articlemngrprjsrch2rf06y was removed from ai.azure.com. That connection had category CognitiveSearch / Azure AI Search and pointed at https://articlemngrprj-srch.search.windows.net/; the underlying Azure AI Search service was not available in the resource group.

After the connection was removed, Search Diagnostics still executed. The page did not report a missing Foundry project connection or setup error. The runtime still showed both scoped retrieval bindings as Foundry vector stores:

• Workspace binding: FoundryVectorStore
• Workspace resource present: Yes
• Project binding: FoundryVectorStore
• Project resource present: Yes
• Ready for scoped retrieval probe: True
• Ready for Responses synthesis: True

Same workspace and question:

• Workspace: BackTheApp.software
• Workspace ID: 019d35a8-4eeb-7a1e-9d5f-040b08e60841
• Question: what is the purpose of this workspace
• Local corpus: 27 digital assets, 4 active projects, 22 conversation threads, 121 conversation turns

Observed staging result after deleting the stale connection:

Component	Status	Elapsed ms	Result
KBA file_search	VectorStoreTimedOutUsedLocalFallback	90007	file_search hit the 90-second scoped retrieval budget; local ContentTraker evidence was used.
KPA file_search	VectorStoreNoMatchUsedLocalFallback	12615	file_search completed within budget and returned no match; local ContentTraker evidence was used.
Answer synthesis	Generated from local fallback evidence after vector timeout	58912	Final answer was generated from local fallback evidence.

This post-removal run argues against the stale Foundry AI Search connection being required by this application path. Removing it did not produce an execution error. It also did not resolve the latency, because KBA still timed out at the 90-second Responses file_search budget.

Azure Monitor for the Azure AI resource over 2026-06-01T18:45Z to 2026-06-01T19:30Z showed:

Model deployment	Operation	Status code	Count
gpt-5.4-pro	create_response	200	10
gpt-5.4-pro	create_response	499	2

No 429 status code appeared in the metric split for this window. The two 499 entries correspond to our application canceling the request at the 90-second timeout.

Azure Support Questions

1. Is 47 to 88 seconds expected for Responses file_search over a small Foundry vector store?
2. Why would the same vector store intermittently time out at 90 seconds and later succeed in 55 seconds?
3. Are the two HTTP 499 records visible to Azure support as server-side work that continued after client cancellation?
4. Is there any hidden service-side throttling, queueing, cold-index warmup, or backend Azure AI Search issue that would not surface as HTTP 429?
5. Are there known latency issues with gpt-5.4-pro plus Responses file_search on project-scoped Foundry vector stores in eastus2?
6. Can Azure support inspect whether the long create_response duration is dominated by file-backed vector retrieval, model inference, capacity queueing, or another internal provider stage?
7. Is an Azure AI Search / CognitiveSearch Foundry project connection required for direct Responses file_search calls that supply tools[].vector_store_ids, or are the Foundry vector stores sufficient for this path?
8. Can Azure support inspect why KBA still timed out after the stale Foundry AI Search connection was removed?
9. Can Azure support inspect these successful request IDs:
   • c42101c2-e6f5-440f-9055-9520287723cf
   • 72445806-6d18-447f-91d4-857175ab305b
   • a87882fa-9ba2-4083-9270-57d9db000b24
   • ae474a7d-fab2-41b7-a484-0b6c799fba59
   • b11e4eaf-9438-4a7d-81f2-2f83a8315f8c
   • f19a3edc-a466-49e6-b318-fa3f5912082c
   • 26992faf-12b1-4c88-a198-7be6feebf6e4

Implementation Details

Configuration

Relevant file: ArticleMngrShared/Configuration/FoundryConfiguration.cs

public sealed class FoundryConfiguration
{
  public string ProjectEndpoint { get; set; } = string.Empty;
  public string ResponsesEndpoint { get; set; } = string.Empty;
  public string ModelDeploymentName { get; set; } = string.Empty;
  public string EmbeddingDeploymentName { get; set; } = "text-embedding-3-large";
  public bool UseRbac { get; set; } = true;

  public bool IsResponsesValid()
  {
    return (HasValidProjectEndpoint() || HasValidResponsesEndpoint())
        && string.IsNullOrWhiteSpace(ModelDeploymentName) == false;
  }

  public Uri GetResponsesEndpointUri()
  {
    if (IsResponsesValid() == false)
    {
      throw new InvalidOperationException("FoundryConfiguration ProjectEndpoint or ResponsesEndpoint and ModelDeploymentName are required for Responses API calls.");
    }

    var endpoint = ResolveResponsesEndpoint();
    return new Uri(endpoint, UriKind.Absolute);
  }

  private string ResolveResponsesEndpoint()
  {
    if (HasValidProjectEndpoint()
        && (HasValidResponsesEndpoint() == false || IsProjectScopedOpenAIEndpoint(ResponsesEndpoint) == false))
    {
      return AppendOpenAIV1(ProjectEndpoint);
    }

    return NormalizeTrailingSlash(ResponsesEndpoint);
  }
}

Container App configuration currently supplies:

FoundryConfiguration__ProjectEndpoint=https://articlemngrprj-resource.services.ai.azure.com/api/projects/articlemngrprj
FoundryConfiguration__ModelDeploymentName=gpt-5.4-pro
FoundryConfiguration__EmbeddingDeploymentName=text-embedding-3-large
FoundryConfiguration__UseRbac=true

Vector Store Setup

Relevant files:

• ArticleMngrShared/Engines/FoundryAgentEngine.cs
• ArticleMngrShared/ResourceAccess/FoundryAgentRA.cs
• ArticleMngrShared/Engines/FoundryVectorStoreEngine.cs

The setup creates vector stores only. It intentionally does not create Persistent Agents.

public async Task<bool> ProvisionResponsesKnowledgeResourcesAsync(
  ContentKeeper contentKeeper,
  string? correlationId = null,
  CancellationToken ct = default)
{
  if (string.IsNullOrEmpty(contentKeeper.FoundryVectorStoreId))
  {
    var vectorStoreName = BuildWorkspaceVectorStoreName(contentKeeper);
    var workspaceVectorStore = await _foundryAgentRA.CreateVectorStoreAsync(vectorStoreName, ct);
    contentKeeper.FoundryVectorStoreId = workspaceVectorStore.Id;
    contentKeeper.RefreshVectorStoreInventories();
  }

  if (string.IsNullOrEmpty(contentKeeper.ProjectFoundryVectorStoreId))
  {
    var vectorStoreName = BuildProjectVectorStoreName(contentKeeper);
    var projectVectorStore = await _foundryAgentRA.CreateVectorStoreAsync(vectorStoreName, ct);
    contentKeeper.ProjectFoundryVectorStoreId = projectVectorStore.Id;
    contentKeeper.RefreshVectorStoreInventories();
  }

  return true;
}

The resource-access layer calls the Azure AI Agents SDK to create the vector store:

public async Task<VectorStore> CreateVectorStoreAsync(
  string name,
  CancellationToken ct = default)
{
  var vectorStore = await _client.VectorStores.CreateVectorStoreAsync(
    name: name,
    cancellationToken: ct);

  return vectorStore.Value;
}

Digital assets are uploaded to the target vector store as files:

public async Task<VectorStoreFile> UploadFileAndAddToVectorStoreAsync(
  string vectorStoreId,
  Stream fileStream,
  string fileName,
  CancellationToken ct = default)
{
  var uploadedFile = await _client.Files.UploadFileAsync(
    fileStream,
    PersistentAgentFilePurpose.Agents,
    fileName,
    ct);

  var vectorStoreFile = await _client.VectorStores.CreateVectorStoreFileAsync(
    vectorStoreId: vectorStoreId,
    fileId: uploadedFile.Value.Id,
    cancellationToken: ct);

  return vectorStoreFile.Value;
}

The engine records the returned vector-store file ID on the digital asset:

using var stream = new MemoryStream(digitalAsset.Content);
var fileName = digitalAsset.SafeFileName ?? digitalAsset.FileName ?? $"{digitalAsset.id}.md";

var vectorStoreFile = await _foundryAgentRA.UploadFileAndAddToVectorStoreAsync(
  targetVectorStoreId,
  stream,
  fileName,
  ct);

digitalAsset.FoundryFileId = vectorStoreFile.Id;
contentKeeper.RefreshVectorStoreInventories();

Responses file_search Invocation

Relevant files:

• ArticleMngrShared/Engines/ScopedKnowledgeRetrievalService.cs
• ArticleMngrShared/ResourceAccess/ResponsesClientRA.cs
• ArticleMngrShared/DTO/AiCompletionDtos.cs

The KBA/KPA retrieval service calls Responses with one vector store ID. The timeout is currently set to 90 seconds by dependency default.

private static readonly TimeSpan DefaultVectorStoreSearchTimeout = TimeSpan.FromSeconds(90);

public ScopedKnowledgeRetrievalService(
  ResponsesClientRA? responsesClientRA = null,
  TimeSpan? vectorStoreSearchTimeout = null,
  ILogger<ScopedKnowledgeRetrievalService>? logger = null)
{
  _responsesClientRA = responsesClientRA;
  _vectorStoreSearchTimeout = vectorStoreSearchTimeout ?? DefaultVectorStoreSearchTimeout;
  _logger = logger;
}

using var timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeoutCts.CancelAfter(_vectorStoreSearchTimeout);

result = await _responsesClientRA.CompleteAsync(new AiCompletionRequest
{
  Instructions = BuildRetrievalProbeInstructions(lane),
  Input = question,
  Options = new AIChatOptions(maxOutputTokenCount: 800, temperature: 0.0f),
  Store = false,
  EndUserId = workspace.IdGuid.ToString(),
  FileSearchVectorStoreIds = [vectorStoreId]
}, timeoutCts.Token);

The retrieval instructions explicitly tell the model to use only file_search results:

private static string BuildRetrievalProbeInstructions(AgentKnowledgeLaneValues lane)
{
  var scope = lane == AgentKnowledgeLaneValues.KPA
    ? "project, conversation thread, and conversation turn knowledge"
    : "workspace digital-asset knowledge";

  return $"""
    You are retrieving evidence for ContentTraker {lane}.
    Scope: {scope}.
    Use file_search results only. Return concise evidence relevant to the user's question.
    If the attached vector store does not contain supporting evidence, say exactly: NO_MATCHING_EVIDENCE.
    """;
}

The transport builds this Responses API payload:

private Dictionary<string, object?> BuildPayload(AiCompletionRequest request)
{
  var options = request.Options ?? new AIChatOptions();

  var payload = new Dictionary<string, object?>
  {
    ["model"] = _foundryConfiguration.ModelDeploymentName,
    ["input"] = request.Input,
    ["instructions"] = string.IsNullOrWhiteSpace(request.Instructions) ? null : request.Instructions,
    ["max_output_tokens"] = options.MaxOutputTokenCount,
    ["store"] = request.Store,
    ["user"] = string.IsNullOrWhiteSpace(request.EndUserId) ? null : request.EndUserId
  };

  var vectorStoreIds = request.FileSearchVectorStoreIds
    .Where(id => string.IsNullOrWhiteSpace(id) == false)
    .Distinct(StringComparer.Ordinal)
    .ToArray();

  if (vectorStoreIds.Length > 0)
  {
    payload["tools"] = new object[]
    {
      new
      {
        type = "file_search",
        vector_store_ids = vectorStoreIds
      }
    };
  }

  return payload;
}

For the failing KBA diagnostic, the effective payload shape is:

{
  "model": "gpt-5.4-pro",
  "input": "what is the purpose of this workspace",
  "instructions": "You are retrieving evidence for ContentTraker KBA...\nUse file_search results only...",
  "max_output_tokens": 800,
  "store": false,
  "user": "019d35a8-4eeb-7a1e-9d5f-040b08e60841",
  "tools": [
    {
      "type": "file_search",
      "vector_store_ids": [
        "vs_uLY7yQIK6ca1iwjqXk1ETgPU"
      ]
    }
  ]
}

The transport sends the request to the project-scoped Responses API endpoint with RBAC bearer token and, in current diagnostics builds, correlation headers:

var endpoint = _foundryConfiguration.GetResponsesEndpointUri();
var responseUri = new Uri(endpoint, "responses");
var payload = BuildPayload(request);

using var httpRequest = new HttpRequestMessage(HttpMethod.Post, responseUri)
{
  Content = JsonContent.Create(payload)
};

var token = await _credential.GetTokenAsync(
  new TokenRequestContext(TokenScopes),
  cancellationToken);

httpRequest.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token.Token);
httpRequest.Headers.TryAddWithoutValidation("x-ms-client-request-id", clientRequestId);
httpRequest.Headers.TryAddWithoutValidation("x-ms-return-client-request-id", "true");

using var httpResponse = await _httpClient.SendAsync(httpRequest, cancellationToken);
var body = await httpResponse.Content.ReadAsStringAsync(cancellationToken);

Timing Metrics and Logs

The application records file_search timing as both logs and OpenTelemetry metrics:

public const string MeterName = "ContentTraker.Search";
public const string FileSearchDurationMetricName = "contenttraker.file_search.duration";
public const string FileSearchSkippedMetricName = "contenttraker.file_search.skipped";

private static readonly Histogram<double> FileSearchDuration = Meter.CreateHistogram<double>(
  FileSearchDurationMetricName,
  unit: "ms",
  description: "Elapsed time for Responses file_search vector-store retrieval probes.");

internal static void RecordFileSearchDuration(
  TimeSpan elapsed,
  AgentKnowledgeLaneValues lane,
  string status,
  TimeSpan timeoutBudget)
{
  var tags = new TagList
  {
    { "tool", "file_search" },
    { "provider", "azure_openai_responses" },
    { "knowledge_lane", lane.ToString() },
    { "status", status },
    { "timeout_budget_ms", timeoutBudget.TotalMilliseconds }
  };

  FileSearchDuration.Record(elapsed.TotalMilliseconds, tags);
}

The web host registers the meter for OpenTelemetry export:

metrics.AddMeter(ScopedKnowledgeRetrievalMetrics.MeterName);

The log line used for support evidence is:

_logger?.LogInformation(
  "Responses file_search probe completed for {KnowledgeLane} with {SearchStatus} in {ElapsedMilliseconds} ms. TimeoutBudgetMs={TimeoutBudgetMilliseconds}; RequestId={RequestId}",
  lane,
  status,
  stopwatch.ElapsedMilliseconds,
  _vectorStoreSearchTimeout.TotalMilliseconds,
  string.IsNullOrWhiteSpace(requestId) ? "none" : requestId);

Diagnostic Commands Used

Log query:

ContainerAppConsoleLogs_CL
| where TimeGenerated > ago(24h)
| where ContainerAppName_s == 'userweb'
| where Log_s contains 'Responses file_search probe completed'
| parse Log_s with * 'completed for ' Lane ' with ' Status ' in ' Elapsed:int ' ms. TimeoutBudgetMs=' Budget:int '; RequestId=' RequestId
| project TimeGenerated, RevisionName_s, Lane, Status, Elapsed, Budget, RequestId, Log_s
| order by TimeGenerated asc

Azure Monitor metric split:

$resourceId = '/subscriptions/af0abafe-ad7b-472c-bf3d-effec675805a/resourceGroups/ArticleMngResources_RG/providers/Microsoft.CognitiveServices/accounts/articlemngrprj-resource'
az monitor metrics list `
  --resource $resourceId `
  --namespace Microsoft.CognitiveServices/accounts `
  --metric AzureOpenAIRequests `
  --start-time 2026-06-01T18:45:00Z `
  --end-time 2026-06-01T19:30:00Z `
  --interval PT1M `
  --aggregation Total `
  --filter "StatusCode eq '*' and OperationName eq '*' and ModelDeploymentName eq '*'"

Current Assessment

The implementation appears to be using Responses file_search against the Foundry vector store ID correctly. This is file-backed vector retrieval managed by Foundry/Responses, not a direct raw Azure AI Search query from ContentTraker. The observed issue is provider-side latency or queueing in the create_response operation when the file_search tool is attached.

The available evidence does not show explicit HTTP 429 throttling. The timeout records appear as HTTP 499, which matches client-side cancellation at our 90-second budget.

The 2026-06-04 provider-stage timing strengthens this assessment: payload build, response body read, and JSON mapping were effectively zero; the dominant successful-call interval was HTTP send-to-response-headers inside the provider call. A warm rerun still timed out, so the issue should not be closed as a simple local app cold start.

The 2026-06-05 post-removal staging run further narrows the issue: deleting the stale Foundry project Azure AI Search connection did not break the application path, but it also did not remove the KBA timeout. The remaining support question is why Responses file_search over the Foundry vector store ID is still taking roughly the full 90-second retrieval budget for a small workspace vector store.

[fahimmrgt]

is Ai 900 a course to do