[dart][dart-dio] Nullable support/improvements (#8727)

* [dart-dio] Disable nullable fields by default

This is not in line with the OAS and will prevent future Dart nullabilty features (NNBD) from being useful as all types would be optional.
Users can still opt-in for this.

* [dart-dio] Properties are nullable when not required AND not nullable

* [dart][dart-dio] Support nullable/required fields

* properties in built_value need to be nullable when they are nullable in OAS or when they are not required in OAS
* built_value does not support serializing `null` values by default as it is based on a serialization format based on iterables/lists and not maps
* dart-dio uses the built_value json plugin to convert the built_value format to regular json
* by generating a custom serializer for each class we can add support for serializing `null` values if the property is required AND nullable in OAS
* this is a breaking change as not all properties in the models are nullable by default anymore

* Implement required/nullable for dart

* Changes for set types and enum names after rebase

* Add some comments and fix built_value fields with default being nullable

* More rebase changes and regenerate docs

Author: kuhnroyal

docs/generators/dart-dio.md

@@ -12,7 +12,7 @@ These options may be applied as additional-properties (cli) or configOptions (pl
 |disallowAdditionalPropertiesIfNotPresent|If false, the 'additionalProperties' implementation (set to true by default) is compliant with the OAS and JSON schema specifications. If true (default), keep the old (incorrect) behaviour that 'additionalProperties' is set to false by default.|<dl><dt>**false**</dt><dd>The 'additionalProperties' implementation is compliant with the OAS and JSON schema specifications.</dd><dt>**true**</dt><dd>Keep the old (incorrect) behaviour that 'additionalProperties' is set to false by default.</dd></dl>|true|
 |ensureUniqueParams|Whether to ensure parameter names are unique in an operation (rename parameters that are not).| |true|
 |legacyDiscriminatorBehavior|Set to true for generators with better support for discriminators. (Python, Java, Go, PowerShell, C#have this enabled by default).|<dl><dt>**true**</dt><dd>The mapping in the discriminator includes descendent schemas that allOf inherit from self and the discriminator mapping schemas in the OAS document.</dd><dt>**false**</dt><dd>The mapping in the discriminator includes any descendent schemas that allOf inherit from self, any oneOf schemas, any anyOf schemas, any x-discriminator-values, and the discriminator mapping schemas in the OAS document AND Codegen validates that oneOf and anyOf schemas contain the required discriminator and throws an error if the discriminator is missing.</dd></dl>|true|
-|nullableFields|Is the null fields should be in the JSON payload| |null|
+|nullableFields|Make all fields nullable in the JSON payload| |null|
 |prependFormOrBodyParameters|Add form or body parameters to the beginning of the parameter list.| |false|
 |pubAuthor|Author name in generated pubspec| |null|
 |pubAuthorEmail|Email address of the author in generated pubspec| |null|

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/DartDioClientCodegen.java

@@ -40,7 +40,7 @@ public class DartDioClientCodegen extends DartClientCodegen {
 
     private static final String CLIENT_NAME = "clientName";
 
-    private boolean nullableFields = true;
+    private boolean nullableFields = false;
     private String dateLibrary = "core";
 
     public DartDioClientCodegen() {
@@ -49,7 +49,7 @@ public DartDioClientCodegen() {
         embeddedTemplateDir = "dart-dio";
         this.setTemplateDir(embeddedTemplateDir);
 
-        cliOptions.add(new CliOption(NULLABLE_FIELDS, "Is the null fields should be in the JSON payload"));
+        cliOptions.add(new CliOption(NULLABLE_FIELDS, "Make all fields nullable in the JSON payload"));
         CliOption dateLibrary = new CliOption(DATE_LIBRARY, "Option. Date library to use").defaultValue(this.getDateLibrary());
         Map<String, String> dateOptions = new HashMap<>();
         dateOptions.put("core", "Dart core library (DateTime)");
@@ -284,6 +284,34 @@ public void postProcessModelProperty(CodegenModel model, CodegenProperty propert
             // enums are generated with built_value and make use of BuiltSet
             model.imports.add("BuiltSet");
         }
+
+        property.getVendorExtensions().put("x-built-value-serializer-type", createBuiltValueSerializerType(property));
+    }
+
+    private String createBuiltValueSerializerType(CodegenProperty property) {
+        final StringBuilder sb = new StringBuilder("const FullType(");
+        if (property.isContainer) {
+            appendCollection(sb, property);
+        } else {
+            sb.append(property.datatypeWithEnum);
+        }
+        sb.append(")");
+        return sb.toString();
+    }
+
+    private void appendCollection(StringBuilder sb, CodegenProperty property) {
+        sb.append(property.baseType);
+        sb.append(", [FullType(");
+        if (property.isMap) {
+            // a map always has string keys
+            sb.append("String), FullType(");
+        }
+        if (property.items.isContainer) {
+            appendCollection(sb, property.items);
+        } else {
+            sb.append(property.items.datatypeWithEnum);
+        }
+        sb.append(")]");
     }
 
     @Override

modules/openapi-generator/src/main/resources/dart-dio/class.mustache

@@ -9,24 +9,110 @@ abstract class {{classname}} implements Built<{{classname}}, {{classname}}Builde
     {{#description}}
     /// {{{description}}}
     {{/description}}
+    {{!
+        A field is @nullable in built_value when it is
+        nullable || (!required && !defaultValue) in OAS
+    }}
     {{#isNullable}}
     @nullable
     {{/isNullable}}
+    {{^isNullable}}
+    {{^required}}
+    {{^defaultValue}}
+    @nullable
+    {{/defaultValue}}
+    {{/required}}
+    {{/isNullable}}
     @BuiltValueField(wireName: r'{{baseName}}')
     {{{datatypeWithEnum}}} get {{name}};
     {{#allowableValues}}
     // {{#min}}range from {{{min}}} to {{{max}}}{{/min}}{{^min}}enum {{name}}Enum { {{#values}} {{{.}}}, {{/values}} };{{/min}}
     {{/allowableValues}}
 
 {{/vars}}
-    // Boilerplate code needed to wire-up generated code
     {{classname}}._();
 
     static void _initializeBuilder({{{classname}}}Builder b) => b{{#vars}}{{#defaultValue}}
         ..{{{name}}} = {{#isEnum}}{{^isContainer}}const {{{enumName}}}._({{/isContainer}}{{/isEnum}}{{{defaultValue}}}{{#isEnum}}{{^isContainer}}){{/isContainer}}{{/isEnum}}{{/defaultValue}}{{/vars}};
 
     factory {{classname}}([void updates({{classname}}Builder b)]) = _${{classname}};
-    static Serializer<{{classname}}> get serializer => _${{#lambda.camelcase}}{{{classname}}}{{/lambda.camelcase}}Serializer;
+
+    @BuiltValueSerializer(custom: true)
+    static Serializer<{{classname}}> get serializer => _${{classname}}Serializer();
+}
+
+{{!
+    Generate a custom serializer in order to support combinations of required and nullable.
+    By default built_value does not serialize null fields.
+}}
+class _${{classname}}Serializer implements StructuredSerializer<{{classname}}> {
+
+    @override
+    final Iterable<Type> types = const [{{classname}}, _${{classname}}];
+    @override
+    final String wireName = r'{{classname}}';
+
+    @override
+    Iterable<Object> serialize(Serializers serializers, {{{classname}}} object,
+        {FullType specifiedType = FullType.unspecified}) {
+        final result = <Object>[];
+        {{#vars}}
+        {{#required}}
+        {{!
+            A required property need to always be part of the serialized output.
+            When it is nullable, null is serialized, otherwise it is an error if it is null.
+        }}
+        result
+            ..add(r'{{baseName}}')
+            ..add({{#isNullable}}object.{{{name}}} == null ? null : {{/isNullable}}serializers.serialize(object.{{{name}}},
+                specifiedType: {{{vendorExtensions.x-built-value-serializer-type}}}));
+        {{/required}}
+        {{^required}}
+        if (object.{{{name}}} != null) {
+            {{! Non-required properties are only serialized if not null. }}
+            result
+                ..add(r'{{baseName}}')
+                ..add(serializers.serialize(object.{{{name}}},
+                    specifiedType: {{{vendorExtensions.x-built-value-serializer-type}}}));
+        }
+        {{/required}}
+        {{/vars}}
+        return result;
+    }
+
+    @override
+    {{classname}} deserialize(Serializers serializers, Iterable<Object> serialized,
+        {FullType specifiedType = FullType.unspecified}) {
+        final result = {{classname}}Builder();
+
+        final iterator = serialized.iterator;
+        while (iterator.moveNext()) {
+            final key = iterator.current as String;
+            iterator.moveNext();
+            final dynamic value = iterator.current;
+            switch (key) {
+                {{#vars}}
+                case r'{{baseName}}':
+                    {{#isContainer}}
+                    result.{{{name}}}.replace(serializers.deserialize(value,
+                        specifiedType: {{{vendorExtensions.x-built-value-serializer-type}}}) as {{{datatypeWithEnum}}});
+                    {{/isContainer}}
+                    {{#isModel}}
+                    result.{{{name}}}.replace(serializers.deserialize(value,
+                        specifiedType: {{{vendorExtensions.x-built-value-serializer-type}}}) as {{{datatypeWithEnum}}});
+                    {{/isModel}}
+                    {{^isContainer}}
+                    {{^isModel}}
+                    result.{{{name}}} = serializers.deserialize(value,
+                        specifiedType: {{{vendorExtensions.x-built-value-serializer-type}}}) as {{{datatypeWithEnum}}};
+                    {{/isModel}}
+                    {{/isContainer}}
+                    break;
+                {{/vars}}
+            }
+        }
+        return result.build();
+    }
 }
 {{!
     Generate an enum for any variables that are declared as inline enums

modules/openapi-generator/src/main/resources/dart-dio/model_test.mustache

@@ -6,7 +6,9 @@ import 'package:test/test.dart';
 // tests for {{{classname}}}
 void main() {
   {{^isEnum}}
-  final instance = {{{classname}}}();
+  {{! Due to required vars without default value we can not create a full instance here }}
+  final instance = {{{classname}}}Builder();
+  // TODO add properties to the builder and call build()
   {{/isEnum}}
 
   group({{{classname}}}, () {

modules/openapi-generator/src/main/resources/dart2/class.mustache

@@ -2,7 +2,11 @@ class {{{classname}}} {
   /// Returns a new [{{{classname}}}] instance.
   {{{classname}}}({
   {{#vars}}
-    {{#required}}{{^defaultValue}}@required {{/defaultValue}}{{/required}}this.{{{name}}}{{^isNullable}}{{#defaultValue}} = {{#isEnum}}{{^isContainer}}const {{{enumName}}}._({{/isContainer}}{{/isEnum}}{{{defaultValue}}}{{#isEnum}}{{^isContainer}}){{/isContainer}}{{/isEnum}}{{/defaultValue}}{{/isNullable}},
+    {{!
+        A field is @required in Dart when it is
+        required && !nullable && !defaultValue in OAS
+    }}
+    {{#required}}{{^isNullable}}{{^defaultValue}}@required {{/defaultValue}}{{/isNullable}}{{/required}}this.{{{name}}}{{^isNullable}}{{#defaultValue}} = {{#isEnum}}{{^isContainer}}const {{{enumName}}}._({{/isContainer}}{{/isEnum}}{{{defaultValue}}}{{#isEnum}}{{^isContainer}}){{/isContainer}}{{/isEnum}}{{/defaultValue}}{{/isNullable}},
   {{/vars}}
   });
 
@@ -39,33 +43,37 @@ class {{{classname}}} {
   Map<String, dynamic> toJson() {
     final json = <String, dynamic>{};
   {{#vars}}
+    {{^required}}
     if ({{{name}}} != null) {
+    {{/required}}
     {{#isDateTime}}
       {{#pattern}}
-      json[r'{{{baseName}}}'] = _dateEpochMarker == '{{{pattern}}}'
+      json[r'{{{baseName}}}'] = {{#required}}{{#isNullable}}{{{name}}} == null ? null : {{/isNullable}}{{/required}}_dateEpochMarker == '{{{pattern}}}'
         ? {{{name}}}.millisecondsSinceEpoch
         : {{{name}}}.toUtc().toIso8601String();
       {{/pattern}}
       {{^pattern}}
-      json[r'{{{baseName}}}'] = {{{name}}}.toUtc().toIso8601String();
+      json[r'{{{baseName}}}'] = {{#required}}{{#isNullable}}{{{name}}} == null ? null : {{/isNullable}}{{/required}}{{{name}}}.toUtc().toIso8601String();
       {{/pattern}}
     {{/isDateTime}}
     {{#isDate}}
       {{#pattern}}
-      json[r'{{{baseName}}}'] = _dateEpochMarker == '{{{pattern}}}'
+      json[r'{{{baseName}}}'] = {{#required}}{{#isNullable}}{{{name}}} == null ? null : {{/isNullable}}{{/required}}_dateEpochMarker == '{{{pattern}}}'
         ? {{{name}}}.millisecondsSinceEpoch
         : _dateFormatter.format({{{name}}}.toUtc());
       {{/pattern}}
       {{^pattern}}
-      json[r'{{{baseName}}}'] = _dateFormatter.format({{{name}}}.toUtc());
+      json[r'{{{baseName}}}'] = {{#required}}{{#isNullable}}{{{name}}} == null ? null : {{/isNullable}}{{/required}}_dateFormatter.format({{{name}}}.toUtc());
       {{/pattern}}
     {{/isDate}}
     {{^isDateTime}}
       {{^isDate}}
-      json[r'{{{baseName}}}'] = {{{name}}};
+      json[r'{{{baseName}}}'] = {{#required}}{{#isNullable}}{{{name}}} == null ? null : {{/isNullable}}{{/required}}{{{name}}};
       {{/isDate}}
     {{/isDateTime}}
+    {{^required}}
     }
+    {{/required}}
   {{/vars}}
     return json;
   }

samples/client/petstore/dart-dio/petstore_client_lib/lib/model/api_response.dart

@@ -24,12 +24,74 @@ abstract class ApiResponse implements Built<ApiResponse, ApiResponseBuilder> {
     @BuiltValueField(wireName: r'message')
     String get message;
 
-    // Boilerplate code needed to wire-up generated code
     ApiResponse._();
 
     static void _initializeBuilder(ApiResponseBuilder b) => b;
 
     factory ApiResponse([void updates(ApiResponseBuilder b)]) = _$ApiResponse;
-    static Serializer<ApiResponse> get serializer => _$apiResponseSerializer;
+
+    @BuiltValueSerializer(custom: true)
+    static Serializer<ApiResponse> get serializer => _$ApiResponseSerializer();
+}
+
+class _$ApiResponseSerializer implements StructuredSerializer<ApiResponse> {
+
+    @override
+    final Iterable<Type> types = const [ApiResponse, _$ApiResponse];
+    @override
+    final String wireName = r'ApiResponse';
+
+    @override
+    Iterable<Object> serialize(Serializers serializers, ApiResponse object,
+        {FullType specifiedType = FullType.unspecified}) {
+        final result = <Object>[];
+        if (object.code != null) {
+            result
+                ..add(r'code')
+                ..add(serializers.serialize(object.code,
+                    specifiedType: const FullType(int)));
+        }
+        if (object.type != null) {
+            result
+                ..add(r'type')
+                ..add(serializers.serialize(object.type,
+                    specifiedType: const FullType(String)));
+        }
+        if (object.message != null) {
+            result
+                ..add(r'message')
+                ..add(serializers.serialize(object.message,
+                    specifiedType: const FullType(String)));
+        }
+        return result;
+    }
+
+    @override
+    ApiResponse deserialize(Serializers serializers, Iterable<Object> serialized,
+        {FullType specifiedType = FullType.unspecified}) {
+        final result = ApiResponseBuilder();
+
+        final iterator = serialized.iterator;
+        while (iterator.moveNext()) {
+            final key = iterator.current as String;
+            iterator.moveNext();
+            final dynamic value = iterator.current;
+            switch (key) {
+                case r'code':
+                    result.code = serializers.deserialize(value,
+                        specifiedType: const FullType(int)) as int;
+                    break;
+                case r'type':
+                    result.type = serializers.deserialize(value,
+                        specifiedType: const FullType(String)) as String;
+                    break;
+                case r'message':
+                    result.message = serializers.deserialize(value,
+                        specifiedType: const FullType(String)) as String;
+                    break;
+            }
+        }
+        return result.build();
+    }
 }
 

samples/client/petstore/dart-dio/petstore_client_lib/lib/model/category.dart

@@ -20,12 +20,64 @@ abstract class Category implements Built<Category, CategoryBuilder> {
     @BuiltValueField(wireName: r'name')
     String get name;
 
-    // Boilerplate code needed to wire-up generated code
     Category._();
 
     static void _initializeBuilder(CategoryBuilder b) => b;
 
     factory Category([void updates(CategoryBuilder b)]) = _$Category;
-    static Serializer<Category> get serializer => _$categorySerializer;
+
+    @BuiltValueSerializer(custom: true)
+    static Serializer<Category> get serializer => _$CategorySerializer();
+}
+
+class _$CategorySerializer implements StructuredSerializer<Category> {
+
+    @override
+    final Iterable<Type> types = const [Category, _$Category];
+    @override
+    final String wireName = r'Category';
+
+    @override
+    Iterable<Object> serialize(Serializers serializers, Category object,
+        {FullType specifiedType = FullType.unspecified}) {
+        final result = <Object>[];
+        if (object.id != null) {
+            result
+                ..add(r'id')
+                ..add(serializers.serialize(object.id,
+                    specifiedType: const FullType(int)));
+        }
+        if (object.name != null) {
+            result
+                ..add(r'name')
+                ..add(serializers.serialize(object.name,
+                    specifiedType: const FullType(String)));
+        }
+        return result;
+    }
+
+    @override
+    Category deserialize(Serializers serializers, Iterable<Object> serialized,
+        {FullType specifiedType = FullType.unspecified}) {
+        final result = CategoryBuilder();
+
+        final iterator = serialized.iterator;
+        while (iterator.moveNext()) {
+            final key = iterator.current as String;
+            iterator.moveNext();
+            final dynamic value = iterator.current;
+            switch (key) {
+                case r'id':
+                    result.id = serializers.deserialize(value,
+                        specifiedType: const FullType(int)) as int;
+                    break;
+                case r'name':
+                    result.name = serializers.deserialize(value,
+                        specifiedType: const FullType(String)) as String;
+                    break;
+            }
+        }
+        return result.build();
+    }
 }