Skip to content

[feat][elixir] use ecto schemas and changesets in generated models #21208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 25 commits into
base: master
Choose a base branch
from

Conversation

efcasado
Copy link
Contributor

@efcasado efcasado commented May 4, 2025

Description

This PR extends the Elixir OpenAPI code generator to use Ecto (embeded) schemas for the generated models.

Rather than relying on plain structs or custom validation code, this approach leverages Ecto's built-in validation and casting capabilities to provide a more powerful and idiomatic solution for API client development in Elixir.

The motivation for this change is three-fold:

  • Ecto is widely adopted within the Elixir ecosystem. By aligning the generated code with this de facto standard, we make the output more familiar and maintainable for Elixir developers. Arguably, working with Ecto.Schema and Ecto.Changeset is more idiomatic than easier to reason about that the custom logic currently used by the Elixir generator (eg. generated deserializer module).

  • Improved validation support out of the box: Ecto's validation functions (e.g., validate_required/2, validate_length/3, etc.) enable robust runtime checks for incoming API data, helping catch mismatches early and making client code safer by default.

  • Extensibility: By using Ecto changesets, developers consuming the generated models have a clear and idiomatic extension point for adding custom validation logic or transformation steps, without having to rework the generated code structure. For example, adding validation for a string field with email format would be as simple as patching the generated model with something along the lines of validate_format(:email, ~r/@/).

As usual, curious to hear what you think about this change @wing328 and @mrmstn. I understand this pull-request is more controversial than previous pull-requests, but I believe it helps us elevate the quality of the clients generated by the Elixir generator, making generated clients more idiomatic (read as in accessible) to the wider Elixir community.

Whilst the public API of the generated client does not change, I understand this change could be considered a breaking change. This said and given that the Elixir generator is still to considered to be an alpha version, maybe we can afford targeting master since this change may get the Elixir generator one step closer to leaving its "alpha" status. What do you think?

I am opening this as a draft because whilst the change is fully functional, I am still working on polishing a few rough edges.

Action Log

  • Better handling of enums
  • Do not generate model files for outer enums
  • Better handling of maps with nested models
  • Better handling of arrays with nested models
  • Handle maps with nested models of any depth
  • Better validation of nested models (just like arrays)
  • Consider adding new/1 function to modules

PR checklist

  • Read the contribution guidelines.
  • Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
  • Run the following to build the project and update samples:
    ./mvnw clean package || exit
    ./bin/generate-samples.sh ./bin/configs/*.yaml || exit
    ./bin/utils/export_docs_generators.sh || exit
    
    (For Windows users, please run the script in Git BASH)
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.
  • File the PR against the correct branch: master (upcoming 7.x.0 minor release - breaking changes with fallbacks), 8.0.x (breaking changes without fallbacks)
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

@@ -872,6 +896,9 @@ private boolean getRequiresHttpcWorkaround() {

class ExtendedCodegenModel extends CodegenModel {
public boolean hasImports;
public List<CodegenProperty> primitiveVars = new ArrayList<>();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not strictly required, but they allow us to keep the model template simpler.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is no longer needed since we are now using Ecto.Schema and and Ecto.Changeset.

@efcasado efcasado marked this pull request as ready for review May 4, 2025 12:38
@@ -36,6 +36,7 @@ defmodule OpenapiPetstore.Mixfile do
defp deps do
[
{:tesla, "~> 1.7"},
{:ecto, "~> 3.12"},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered using TypedEctoSchema, which would have resulted in more compact models - but I figured it's best to stick to the most common and standard libraries. Since models are automatically generated, I feel TypedEctoSchema main value proposition does not apply.

@efcasado efcasado marked this pull request as draft May 4, 2025 14:18
embedded_schema do
field :uuid, :string
field :dateTime, :utc_datetime
embeds_one :map, OpenapiPetstore.Model.map()
Copy link
Contributor Author

@efcasado efcasado May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a bug here. It should read as

embeds_one :map, OpenapiPetstore.Model.Animal

Even the type spec (which wasn't change) looks wrong, though. It should read as

  @type t :: %__MODULE__{
    :uuid => String.t | nil,
    :dateTime => DateTime.t | nil,
    :map => OpenapiPetstore.Model.Animal.t | nil
  }

unless I am mistaken.

@spec changeset(t(), map()) :: Ecto.Changeset.t()
def changeset(%__MODULE__{} = struct, params) do
struct
|> Ecto.Changeset.cast(params, [:ArrayArrayNumber])
Copy link
Contributor Author

@efcasado efcasado May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me share an example of Ecto.Changeset.cast in action

alias OpenapiPetstore.Model.ArrayOfArrayOfNumberOnly

iex(1)> ArrayOfArrayOfNumberOnly.changeset(%ArrayOfArrayOfNumberOnly{}, %{"ArrayArrayNumber": [[1.0]]})
#Ecto.Changeset<
  action: nil,
  changes: %{ArrayArrayNumber: [[1.0]]},
  errors: [],
  data: #OpenapiPetstore.Model.ArrayOfArrayOfNumberOnly<>,
  valid?: true,
  ...
>

iex(2)> ArrayOfArrayOfNumberOnly.changeset(%ArrayOfArrayOfNumberOnly{}, %{"ArrayArrayNumber": [["wrong type"]]})
#Ecto.Changeset<
  action: nil,
  changes: %{},
  errors: [
    ArrayArrayNumber: {"is invalid",
     [type: {:array, {:array, :float}}, validation: :cast]}
  ],
  data: #OpenapiPetstore.Model.ArrayOfArrayOfNumberOnly<>,
  valid?: false,
  ...
>

iex(3)> ArrayOfArrayOfNumberOnly.changeset(%ArrayOfArrayOfNumberOnly{}, %{"ArrayArrayNumber": "wrong type"})
#Ecto.Changeset<
  action: nil,
  changes: %{},
  errors: [
    ArrayArrayNumber: {"is invalid",
     [type: {:array, {:array, :float}}, validation: :cast]}
  ],
  data: #OpenapiPetstore.Model.ArrayOfArrayOfNumberOnly<>,
  valid?: false,
  ...
>

Given the valid example from above

%ArrayOfArrayOfNumberOnly{}
|> ArrayOfArrayOfNumberOnly.changeset(%{"ArrayArrayNumber": [[1.0]]})
|> Ecto.Changeset.apply_action!(:insert)
|> JSON.encode!

# => "{\"ArrayArrayNumber\":[[1.0]]}"

|> Deserializer.deserialize(:category, :struct, OpenapiPetstore.Model.Category)
|> Deserializer.deserialize(:tags, :list, OpenapiPetstore.Model.Tag)
@spec changeset(t(), map()) :: Ecto.Changeset.t()
def changeset(%__MODULE__{} = struct, params) do
Copy link
Contributor Author

@efcasado efcasado May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is an example to illustrate what working with embeds looks like

alias OpenapiPetstore.Model.Pet

%Pet{}
|> Pet.changeset(%{name: "John", photoUrls: ["http://picture.me/john"], category: %{id: 1, name: "dogs"}})
|> Ecto.Changeset.apply_action!(:insert)

# => %OpenapiPetstore.Model.Pet{
#            id: nil,
#            category: %OpenapiPetstore.Model.Category{id: 1, name: "dogs"},
#            name: "John",
#            photoUrls: ["http://picture.me/john"],
#            tags: [],
#            status: nil
#        }

where we can see that category was correctly casted to OpenapiPetstore.Model.Category.

@@ -25,25 +24,28 @@ defmodule DeserializerTest do
"name": "sea"
}
],
"status": "foo"
"status": "available"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous version did not validate enum values. This is one of the nice goodies we get from switching to Ecto.

%FormatTest{
integer: 1,
int32: 2,
int64: 3,
number: 4.1,
float: 5.2,
double: 6.3,
decimal: "7.4",
decimal: 7.4,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous version did not handle type-casting of decimal values. This is a nice goodie we get from switching to Ecto.

@@ -6,6 +6,7 @@ defmodule OuterEnumTest do

@valid_json """
{
"enum_string_required": "lower",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shows that the previous implementation was "off", and shows one of the benefits of migrating to Ecto. Now, it's easier and more idiomatic to write proper type-casting and validations, which allows us to have a better Elixir generator.

@efcasado efcasado force-pushed the feat-elixir-ecto-schemas branch 4 times, most recently from 22d79db to aec6317 Compare May 8, 2025 18:14
@efcasado efcasado force-pushed the feat-elixir-ecto-schemas branch from aec6317 to d994f96 Compare May 8, 2025 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant