Skip to content

feat(schema): Implement Schema Migration System for ZIO Schema 2#1173

Open
vimalinx wants to merge 1 commit intozio:mainfrom
vimalinx:feature/schema-migration-system
Open

feat(schema): Implement Schema Migration System for ZIO Schema 2#1173
vimalinx wants to merge 1 commit intozio:mainfrom
vimalinx:feature/schema-migration-system

Conversation

@vimalinx
Copy link

@vimalinx vimalinx commented Mar 5, 2026

Summary

Implements a pure algebraic migration system for ZIO Schema 2, addressing Issue #519 ($4,000 bounty).

This PR provides a comprehensive solution for representing structural transformations between schema versions as first-class, serializable data.

Core Components

1. MigrationError

  • Comprehensive error handling with path information
  • Multiple error types: FieldNotFound, CaseNotFound, TypeMismatch, TransformFailed, etc.
  • Precise diagnostics: "Failed to apply TransformValue at .addresses.each.streetNumber"

2. MigrationAction ADT

Pure data representation of structural transformations:

Action Description Reverse
AddField Add field with default DropField
DropField Remove field AddField
RenameField Rename field Rename back
TransformValue Transform field value Best-effort
MandateField Make required OptionalizeField
OptionalizeField Make optional MandateField
ChangeFieldType Type conversion Best-effort
JoinFields Combine fields SplitField
RenameCase Rename variant case Rename back
TransformCase Nested case migration Reversed actions
TransformElements Transform collection Best-effort
TransformKeys/Values Transform map Best-effort

3. DynamicMigration

  • Schema-less migration operating on DynamicValue
  • Composable with ++ operator
  • Reversible with .reverse
  • Serializable

4. Migration[A, B]

  • Type-safe wrapper with source/target schemas
  • Compile-time type guarantees
  • Composable migrations

5. MigrationBuilder[A, B]

  • Fluent DSL for constructing migrations
  • Extension methods on DynamicOptic

Key Features

Pure Data: No functions, closures, or reflection
Serializable: All migrations can be serialized to JSON, MessagePack, etc.
Type-Safe: Migration[A, B] provides compile-time type guarantees
Bidirectional: Most migrations can be reversed
Composable: Migrations can be composed with ++ operator
Inspectable: Migration structure can be analyzed and debugged

Laws

The system obeys the following algebraic laws:

  1. Identity: Migration.identity[A].apply(a) == Right(a)
  2. Associativity: (m1 ++ m2) ++ m3 == m1 ++ (m2 ++ m3)
  3. Structural Inverse: m.reverse.reverse == m

Usage Example

case class UserV1(name: String, age: Int)
case class UserV2(fullName: String, age: Int, email: String)

val migration: Migration[UserV1, UserV2] = Migration
  .builder[UserV1, UserV2]
  .renameField("name", "fullName")
  .addField("email", SchemaExpr.Literal("", Schema[String]))
  .build

val v1 = UserV1("Alice", 30)
val v2: Either[MigrationError, UserV2] = migration(v1)
// Right(UserV2("Alice", 30, ""))

Files Changed

  • schema/shared/src/main/scala/zio/blocks/schema/migration/

    • MigrationError.scala - Error handling (192 lines)
    • MigrationAction.scala - Action ADT (282 lines)
    • DynamicMigration.scala - Core migration logic (487 lines)
    • Migration.scala - Typed wrapper (159 lines)
    • MigrationBuilder.scala - Fluent DSL (288 lines)
    • MigrationSchema.scala - Schemas for serialization (359 lines)
    • MigrationActionSchema.scala - Action schemas (614 lines)
    • package.scala - Extension methods (98 lines)
  • schema/shared/src/test/scala/zio/blocks/schema/migration/

    • MigrationSpec.scala - Comprehensive tests
  • schema-examples/src/main/scala/zio/blocks/schema/migration/examples/

    • MigrationExample.scala - Usage examples
  • docs/migration/README.md - Documentation

Total: ~2,500 lines of production code + tests

Design Decisions

  1. Pure Data over Functions: Migrations contain no closures, making them serializable
  2. DynamicValue as Core: Leverages existing DynamicValue infrastructure
  3. SchemaExpr for Values: Uses SchemaExpr for value transformations (no reflection)
  4. Path-based Errors: All errors include DynamicOptic paths for debugging
  5. Best-effort Reverse: Some transformations (like TransformValue) have identity reverses

Future Enhancements

  1. Macro-based selector expressions for type-safe field access
  2. Schema validation to ensure migrations are compatible
  3. Migration registry for storing/retrieving migrations by version
  4. Auto-migration generation from schema diffs

Testing

  • Unit tests for all action types
  • Composition and associativity tests
  • Error handling tests
  • Path navigation tests

Fixes #519

…#519)

Implements a pure algebraic migration system for representing structural
transformations between schema versions.

## Core Components

- **MigrationError**: Error handling with path information
- **MigrationAction ADT**: Structural transformation operations
  - AddField, DropField, RenameField
  - TransformValue, ChangeFieldType
  - MandateField, OptionalizeField
  - JoinFields, SplitField
  - RenameCase, TransformCase
  - TransformElements, TransformKeys, TransformValues
- **DynamicMigration**: Schema-less migration on DynamicValue
- **Migration[A, B]**: Typed migration wrapper
- **MigrationBuilder[A, B]**: Fluent DSL builder

## Key Features

- Pure data (no functions, closures, reflection)
- Fully serializable
- Type-safe with Migration[A, B]
- Bidirectional (reverse migrations)
- Composable with ++ operator
- Inspectable and debuggable

## Additional Files

- Schema definitions for serialization (MigrationSchema, MigrationActionSchema)
- Comprehensive test suite (MigrationSpec)
- Documentation (docs/migration/README.md)
- Examples (MigrationExample.scala)

Addresses GitHub Issue zio#519 ($4,000 bounty)
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schema Migration System for ZIO Schema 2

2 participants