GH-5884: Reuse JsonSchemaGenerator in BeanOutputConverter

[ChoMinGi]

Fixes #5884

Replace BeanOutputConverter's internal victools schema generation with JsonSchemaGenerator.generateForType() delegation to ensure consistent JSON Schema behavior between tool calling and structured outputs.

• Add RESPECT_JSONPROPERTY_ORDER and conditional KotlinModule to JsonSchemaGenerator
• Remove duplicate victools setup from BeanOutputConverter
• Remove postProcessSchema(JsonNode) extension point
• Replace it with a protected generateSchema() method as the new extension point — subclasses can post-process by calling super.generateSchema() or replace the generation logic entirely
• Schema formatting is now delegated to JsonSchemaGenerator (Jackson's default pretty printer)

Behavior changes:

• @JsonProperty without explicit required = true no longer forced into required array
• Kotlin nullable fields (String?) no longer forced into required
• Unannotated fields remain required by default (unchanged)

This also addresses the root cause behind #5797 and #5341, as both stem from BeanOutputConverter maintaining a separate schema generation setup.

Open question(resolved)

• The required behavior changes for fields annotated with @JsonProperty (without explicit required = true). Is this acceptable, or should backward compatibility be preserved?
• Resolved: accepted as bug fix. Documented in 2.0.0-M6 upgrade notes.

[sdeleuze]

@ThomasVitale Could your please review this PR?

[ThomasVitale]

I'll have a look later today, thanks

[ThomasVitale]

I'm a bit unsure about this part. I understand it's meant to maintain support for postProcessSchema(). I'm thinking if we should take this opportunity for the 2.0.0 release to remove the postProcessSchema() entirely. Until now it was necessary because the entire JSON Schema generation logic was included in this class and we needed a way to make it customizable that wouldn't require re-inventing the whole logic. And I know it's been used to work around some of the issues in the logic (e.g. the cumbersome handling of required fields or missing support for annotations) which this PR solved once and for all.

Now the whole logic is one call to JsonSchemaGenerator.generateForType(this.type). We could remove the postProcessSchema() method, make the BeanOutputConverter.generateSchema() method protected, and identify that one as the extension point to customize the JSON Schema generation logic. That would make this even more flexible since developers would still have the option to apply post-processing logic, but they would also have the option to completely change the generation logic if they wanted to.

My main concern with the suggested solution is the extra Jackson deserialisation+serialization operations that are always performed, even though there's no post-processing logic defined (which is probably most of the times). Jackson operations are not cheap. With the additional risk of exceptions being thrown performing an operation that is often not needed.

Thoughts? @ChoMinGi @sdeleuze

[ChoMinGi]

@ThomasVitale
Thanks for raising this. I agree with that direction.

I kept the deserialize/post-process/serialize flow only to preserve the existing postProcessSchema() extension point.
But since this is for 2.0.0 and schema generation is now delegated to JsonSchemaGenerator.generateForType(this.type), removing postProcessSchema() and making generateSchema() the protected extension point seems cleaner and more flexible.

That keeps the same construction-time extension lifecycle as today, while avoiding the extra Jackson roundtrip in the default path. Subclasses can still post-process the generated schema by calling super.generateSchema(), or replace the generation logic entirely.

I'll update the PR accordingly and add the migration note under a new Upgrading to 2.0.0-M6 section in upgrade-notes.adoc.

[ChoMinGi]

Updated in the latest push.

postProcessSchema() has been removed, generateSchema() is now protected, and BeanOutputConverter now uses the JsonSchemaGenerator.generateForType(this.type) output directly without the Jackson deserialize/serialize roundtrip.

I also updated the affected schema format assertions, removed the obsolete line separator tests, and added the 2.0.0-M6 upgrade notes for the required-property behavior changes and the migration from postProcessSchema() to generateSchema().

Tests: BeanOutputConverterTest, JsonSchemaGeneratorTests, and KotlinBeanOutputConverterTests pass locally.

[ThomasVitale]

@ChoMinGi thanks so much for the PR. About your open question.

The JsonSchemaGenerator was configured from the start to treat all properties as required by default (to ensure correct behaviour across different model providers), unless a property is annotated explicitly to be optional via any of these annotations:

• @ToolParam(required = false)
• @JsonProperty(required = false)
• @Schema(required = false)
• @Nullable

The logic handling that is in the SpringAiSchemaModule class used internally by the JsonSchemaGenerator.

I think it's correct to apply this same logic to structured output conversion.

• I see as a bug the fact that Kotlin nullable fields were treated as required, so it's great that's now fixed by doing this consolidation.
• I also see as a bug that if I had a field annotated with @JsonProperty (whose required value is false by default if unspecified) was actually treated as required. Now, we are respecting the annotation value, and also reducing the element of surprise.

So I think that's an acceptable change as it's actually about fixing existing bugs in the BeanOutputConverter implementation. @sdeleuze what do you think?

Still, we're changing the behaviour, so it's important to make people aware of it. In the context of this PR, I would suggest mentioning the two behaviour differences in the upgrades notes for the upcoming release here in a new "Upgrading to 2.0.0-M6" section.

And then I would perhaps suggest creating a separate task for reviewing and consolidating the documentation about JSON Schema generation between Tool Calling and Structured Output Conversion.

[ChoMinGi]

@ThomasVitale
Thanks for the clarification on the required-property semantics.
Agreed it's effectively a bug fix, and good to have structured output conversion aligned with JsonSchemaGenerator.
The two behavior differences are documented in the new "Upgrading to 2.0.0-M6"section.

I agree the JSON Schema documentation consolidation between Tool Calling and Structured Output Conversion is better handled as a follow-up. I can open a separate issue for that.

[ChoMinGi]

Follow-up issue created: #5915

[KoreaNirsa]

This appears to introduce a behavioral change in how additionalProperties: false is applied

Previously, BeanOutputConverter used Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT, which enforced additionalProperties: false across all generated object schemas, including nested objects and array items

With the current delegation to JsonSchemaGenerator.generateForType(), additionalProperties: false seems to be applied only to the root schema node, leaving nested object schemas more permissive

This is visible in the updated test expectations where items schemas no longer include additionalProperties: false

I think this may effectively relax schema strictness for nested objects, which could impact

• structured output prompting
• provider-side strict schema handling
• consumers relying on stricter nested JSON Schema output

If this change is intentional, it would be helpful to document it in the upgrade notes