Unify nullable enum handling across exporters

[numbata]

Summary

Define and implement consistent handling for nil enum members across the OAS2, OAS3, and OAS3.1 exporters, including parameters, $ref wrappers, and composition schemas.

Root cause

normalize_enum is implemented independently in three exporter classes. The copies share the same coercion behavior but have diverged on whether a nil enum member is retained. That decision currently depends on local knowledge of the exporter and call path rather than on a single representation-aware policy.

This creates several edge cases:

• OAS2 schemas may preserve nil under the x-nullable extension while OAS3 drops it under EXTENSION.
• OAS3 KEYWORD + $ref can place nil in an enum on an allOf wrapper around a referenced scalar schema. This is pragmatically useful to some code generators, but is not strictly JSON Schema-valid because the referenced schema remains non-nullable.
• OAS3 TYPE_ARRAY can express nullability for inline schemas, and for composition nodes the exporter can wrap the composition in anyOf: [..., {"type": "null"}]; enum handling does not currently recognize that wrapper and may remove nil unnecessarily.
• OAS2 parameters are a separate case: query, path, header, and form values cannot carry JSON null, so preserving nil there produces an invalid Swagger 2.0 parameter enum.
• The original coercion bug was more severe than merely dropping nil: nil.to_i and nil.to_f can fabricate 0 and 0.0 in numeric enums.

There is also no end-to-end test proving that a real Grape introspector path, such as grape-entity documentation with values: and nullable: true, produces the model shape consumed by these exporter branches.

Suggested approaches

1. Extract shared enum normalization logic

Move the common coercion and deduplication logic into an exporter concern or support object. Pass an explicit policy/context describing whether the target location can express nullability, rather than passing schema.nullable directly.

Possible inputs to that policy include:

• OAS version and nullable strategy.
• Whether the value is an inline schema, $ref metadata wrapper, composition wrapper, or non-body parameter.
• Whether the emitted schema actually contains a valid null representation.

This is the preferred long-term approach because it prevents the OAS2/OAS3 copies from drifting again.

2. Make the null-preservation predicate representation-aware

Retain nil only when the emitted schema can represent it:

• OAS3 KEYWORD: nullable: true can accompany the enum, subject to $ref compatibility policy.
• OAS3/OAS3.1 TYPE_ARRAY: retain it when the effective emitted type includes "null", including the composition anyOf wrapper case.
• OAS2 non-body parameters: always drop nil.
• OAS2 schemas and the EXTENSION strategy require an explicit compatibility decision because x-nullable is vendor-specific rather than standard JSON Schema nullability.

The predicate should also preserve enum order and avoid silently changing empty-enum behavior without documentation or tests.

3. Add coverage at both exporter and integration levels

Add focused unit tests for:

• Inline, $ref, allOf, oneOf, and anyOf schemas.
• OAS3 KEYWORD, EXTENSION, and TYPE_ARRAY strategies.
• OAS2 body schemas versus non-body parameters.
• Nil-only, mixed, duplicate, and empty enums.
• Numeric enums containing nil, ensuring no fabricated 0 or 0.0 values.

Add at least one integration test that starts with a real Grape API/introspector declaration and verifies the generated OAS output.

Nuances and tradeoffs

• $ref metadata wrappers are inherently awkward: allOf is conjunctive, so adding an enum containing null does not make a referenced scalar schema nullable. Strict spec correctness would drop nil unless the referenced definition itself is nullable. Some code-generation tools nevertheless honor the wrapper-level nullable keyword, so changing the current OAS3 KEYWORD + $ref behavior may be a compatibility break.
• x-nullable is not a standard JSON Schema keyword. Preserving nil alongside it may match vendor tooling, while dropping nil is more defensible for standards-based validators. The project should choose and document one policy rather than silently applying different policies per exporter.
• OAS2 nullable behavior is already underspecified in the project. This issue should distinguish body/model schemas from non-body parameters instead of applying one blanket OAS2 rule.
• Empty enum: [] is currently omitted in some paths. That may be the right output because an empty enum is not useful, but it is a user-visible behavior change and should be consciously documented and tested.
• A shared concern reduces duplication but increases coupling between exporters. Keep OAS-version-specific representation decisions at the call site or behind a small policy interface; do not make the concern emit version-specific schema keywords.

Non-goals

• Runtime request validation.
• A broad exporter rewrite unrelated to enum/nullability behavior.
• Changing the current PR solely to resolve the pragmatic OAS3 KEYWORD + $ref compatibility tradeoff.

Acceptance criteria

• One shared, tested definition of enum coercion and nil handling exists, or the deliberate reason for retaining separate implementations is documented.
• No exporter emits fabricated numeric enum values from nil.
• Non-body OAS2 parameters never emit JSON null enum members.
• Null preservation is consistent with the selected OAS representation and documented compatibility exceptions.
• Inline and composition nullability paths have regression tests.
• At least one real introspector-to-exporter integration test covers nullable enum values.