fix: ordered json_schema response #2436

ranadeepsingh · 2025-10-14T16:42:00Z

Related Issues/PRs

feat: Add json_schema as responseFormat option #2428

What changes are proposed in this pull request?

Ensure json_schema response is ordered the way as specified. Ordering was being lost in the python dict -> scala Map conversion. Using ListMap instead now. Ordering of json_schema is important in cases like "chain of thought" prompting where a user might want the model to write "reason" before it gives "answer"
Let users only specify the schema part of json_schema and not type out the whole verbose dict/Map
Adding tests to ensure responseFormat output is ordered

How is this patch tested?

I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

Does this PR change any dependencies?

No. You can skip this section.
Yes. Make sure the dependencies are resolved correctly, and list changes here.

Does this PR add a new feature? If so, have you added samples on website?

No. You can skip this section.
Yes. Make sure you have added samples following below steps.

Find the corresponding markdown file for your new feature in website/docs/documentation folder.
Make sure you choose the correct class estimators/transformers and namespace.
Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
Make sure the DocTable points to correct API link.
Navigate to website folder, and run yarn run start to make sure the website renders correctly.
Don't forget to add  before each python code blocks to enable auto-tests for python samples.
Make sure the WebsiteSamplesTests job pass in the pipeline.

ranadeepsingh · 2025-10-14T16:42:10Z

/azp run

azure-pipelines · 2025-10-14T16:42:21Z

Azure Pipelines successfully started running 1 pipeline(s).

codecov-commenter · 2025-10-14T16:58:00Z

Codecov Report

❌ Patch coverage is 83.33333% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.46%. Comparing base (7aac49e) to head (6f72f1d).

Files with missing lines	Patch %	Lines
...napse/ml/services/openai/ResponseFormatUtils.scala	81.81%	10 Missing ⚠️
...ft/azure/synapse/ml/param/JsonEncodableParam.scala	0.00%	4 Missing ⚠️
...oft/azure/synapse/ml/param/UntypedArrayParam.scala	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2436      +/-   ##
==========================================
+ Coverage   84.41%   84.46%   +0.05%     
==========================================
  Files         333      334       +1     
  Lines       17550    17591      +41     
  Branches     1580     1603      +23     
==========================================
+ Hits        14815    14859      +44     
+ Misses       2735     2732       -3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ranadeepsingh · 2025-10-14T17:37:45Z

/azp run

azure-pipelines · 2025-10-14T17:38:00Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-14T17:38:10Z

/azp run

azure-pipelines · 2025-10-14T17:38:21Z

Azure Pipelines successfully started running 1 pipeline(s).

…psingh/SynapseML into rana/ordered_json_schema

ranadeepsingh · 2025-10-14T18:42:16Z

/azp run

azure-pipelines · 2025-10-14T18:42:28Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-14T18:42:37Z

/azp run

azure-pipelines · 2025-10-14T18:42:49Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-14T18:55:46Z

/azp run

azure-pipelines · 2025-10-14T18:55:57Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-14T19:19:27Z

/azp run

azure-pipelines · 2025-10-14T19:19:39Z

Azure Pipelines successfully started running 1 pipeline(s).

...ive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIChatCompletion.scala

mhamilton723

Minor things

...src/test/scala/com/microsoft/azure/synapse/ml/services/openai/ResponseFormatOrderSuite.scala

core/src/main/scala/com/microsoft/azure/synapse/ml/codegen/Wrappable.scala

core/src/main/scala/com/microsoft/azure/synapse/ml/param/JsonEncodableParam.scala

ranadeepsingh · 2025-10-14T23:33:36Z

/azp run

azure-pipelines · 2025-10-14T23:33:47Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-23T14:39:57Z

/azp run

azure-pipelines · 2025-10-23T14:40:09Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-23T15:32:07Z

/azp run

azure-pipelines · 2025-10-23T15:32:19Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-23T15:38:12Z

/azp run

azure-pipelines · 2025-10-23T15:38:24Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-23T16:28:00Z

/azp run

azure-pipelines · 2025-10-23T16:28:12Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-23T16:34:35Z

/azp run

azure-pipelines · 2025-10-23T16:34:46Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-23T17:51:27Z

/azp run

azure-pipelines · 2025-10-23T17:51:38Z

Azure Pipelines successfully started running 1 pipeline(s).

This reverts commit c6aee7d.

ranadeepsingh · 2025-10-23T18:14:26Z

/azp run

azure-pipelines · 2025-10-23T18:14:38Z

Azure Pipelines successfully started running 1 pipeline(s).

ranadeepsingh · 2025-10-23T19:56:03Z

/azp run

azure-pipelines · 2025-10-23T19:56:14Z

Azure Pipelines successfully started running 1 pipeline(s).

BrendanWalsh · 2025-10-23T23:21:43Z

...tive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/ResponseFormatUtils.scala

+      )
+      if (value.contains("strict")) base + ("strict" -> value("strict")) else base
+    } else {
+      val js = value("json_schema").asInstanceOf[Map[String, Any]]


Are we checking the type of json_schema upstream before we cast here? Consider pattern match instead

BrendanWalsh · 2025-10-23T23:26:41Z

...tive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/ResponseFormatUtils.scala

+    val hasSchema = value.contains("schema")
+    if (!hasName || !hasSchema) {
+      throw new IllegalArgumentException(
+        "Bare schema Map must include 'name' and 'schema' keys.")


nit: consider indicating which keys have been specified and which are missing

BrendanWalsh · 2025-10-23T23:29:52Z

cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIResponses.scala

-      js.get("strict").map(v => base ++ Map("strict" -> v)).getOrElse(base)
-    }
-  }
+  // Validation helpers moved into ResponseFormatUtils


nit: unnecessary

BrendanWalsh · 2025-10-23T23:38:45Z

...tive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/ResponseFormatUtils.scala

+    if (hasFlat) {
+      val base = Map(
+        "type" -> "json_schema",
+        "name" -> value("name"),


validation for name + schema?

BrendanWalsh · 2025-10-23T23:50:39Z

...tive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/ResponseFormatUtils.scala

+      if (value.contains("strict")) base + ("strict" -> value("strict")) else base
+    } else {
+      val js = value("json_schema").asInstanceOf[Map[String, Any]]
+      val name = js.getOrElse("name",


does this validate that name/schema is nonempty/nonnull or just that the keys exist?

BrendanWalsh · 2025-10-23T23:59:13Z

...tive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/ResponseFormatUtils.scala

+      throw new IllegalArgumentException(
+        "json_schema requires nested 'json_schema' or top-level 'name' and 'schema'.")
+    }
+    if (hasFlat) {


Should we error/alert if the user erroneously specifies both json_schema + name/schema?

BrendanWalsh · 2025-10-24T00:07:20Z

...ive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIChatCompletion.scala

+    val payload =
+      if (normalized.get("type").exists(_.toString.equalsIgnoreCase("json_schema"))) {
+        Map(
+          "type" -> "json_schema",


nit: consider elevating "json_schema" "name" and "schema" to vals to reduce duplication and avoid potential typo errors if any of these strings need to be updated in the future

working ordered json_schema

8c1016b

ranadeepsingh requested a review from mhamilton723 as a code owner October 14, 2025 16:42

Add responseformat order check test

e485c94

Merge branch 'master' into rana/ordered_json_schema

6c6c038

ranadeepsingh added 2 commits October 14, 2025 11:41

remove line

a6d8f9f

Merge branch 'rana/ordered_json_schema' of https://github.com/ranadee…

a52e935

…psingh/SynapseML into rana/ordered_json_schema

Merge branch 'master' into rana/ordered_json_schema

6f5a271

black style fix

55016db

Simplify listmap usage

fc6579e

mhamilton723 reviewed Oct 14, 2025

View reviewed changes

...ive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIChatCompletion.scala Outdated Show resolved Hide resolved

mhamilton723 reviewed Oct 14, 2025

View reviewed changes

ranadeepsingh added 2 commits October 14, 2025 15:59

style updates

d5233d5

reworked ordering tests

87e38fc

Make schema api independant

eef4051

ranadeepsingh added 2 commits October 23, 2025 08:27

Fix for openai unittests

906f2aa

update init method of TestResponseFormatOrder

ee8f8c1

scala style fix

e129c00

remove unrequired override and some test fixes

df3ea21

scala style fix

ffdaf9d

Update to be in-line model deployment name

c6aee7d

ranadeepsingh added 2 commits October 23, 2025 11:04

Revert "Update to be in-line model deployment name"

1203d34

This reverts commit c6aee7d.

update tests

40303f7

combine json_flattening logic

6f72f1d

ranadeepsingh requested a review from mhamilton723 October 23, 2025 20:37

mhamilton723 approved these changes Oct 23, 2025

View reviewed changes

BrendanWalsh approved these changes Oct 24, 2025

View reviewed changes

ranadeepsingh merged commit fc6675d into microsoft:master Oct 24, 2025
69 of 70 checks passed

fix: ordered json_schema response #2436

fix: ordered json_schema response #2436

Uh oh!

Conversation

ranadeepsingh commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues/PRs

What changes are proposed in this pull request?

How is this patch tested?

Does this PR change any dependencies?

Does this PR add a new feature? If so, have you added samples on website?

Uh oh!

ranadeepsingh commented Oct 14, 2025

Uh oh!

azure-pipelines bot commented Oct 14, 2025

Uh oh!

codecov-commenter commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ranadeepsingh commented Oct 14, 2025

Uh oh!

azure-pipelines bot commented Oct 14, 2025

Uh oh!

ranadeepsingh commented Oct 14, 2025

Uh oh!

azure-pipelines bot commented Oct 14, 2025

Uh oh!

ranadeepsingh commented Oct 14, 2025

Uh oh!

azure-pipelines bot commented Oct 14, 2025

Uh oh!

ranadeepsingh commented Oct 14, 2025

Uh oh!

azure-pipelines bot commented Oct 14, 2025

Uh oh!

ranadeepsingh commented Oct 14, 2025

Uh oh!

azure-pipelines bot commented Oct 14, 2025

Uh oh!

ranadeepsingh commented Oct 14, 2025

Uh oh!

azure-pipelines bot commented Oct 14, 2025

Uh oh!

Uh oh!

mhamilton723 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ranadeepsingh commented Oct 14, 2025

Uh oh!

azure-pipelines bot commented Oct 14, 2025

Uh oh!

ranadeepsingh commented Oct 23, 2025

Uh oh!

azure-pipelines bot commented Oct 23, 2025

Uh oh!

ranadeepsingh commented Oct 23, 2025

Uh oh!

azure-pipelines bot commented Oct 23, 2025

Uh oh!

ranadeepsingh commented Oct 23, 2025

Uh oh!

azure-pipelines bot commented Oct 23, 2025

Uh oh!

ranadeepsingh commented Oct 23, 2025

Uh oh!

azure-pipelines bot commented Oct 23, 2025

Uh oh!

ranadeepsingh commented Oct 23, 2025

Uh oh!

azure-pipelines bot commented Oct 23, 2025

Uh oh!

ranadeepsingh commented Oct 23, 2025

Uh oh!

azure-pipelines bot commented Oct 23, 2025

Uh oh!

ranadeepsingh commented Oct 23, 2025

ranadeepsingh commented Oct 14, 2025 •

edited

Loading

codecov-commenter commented Oct 14, 2025 •

edited

Loading