Skip to content

StackOverflowError in AvroSchemaCodec.encodeToApacheAvro for recursive sealed traits #985

@EtaCassiopeia

Description

@EtaCassiopeia

Description

AvroSchemaCodec.encodeToApacheAvro causes a StackOverflowError when encoding a schema that contains a recursive sealed trait pattern - specifically when a case class within a sealed trait contains a List[SealedTrait] (or similar collection) referring back to the parent sealed trait.

This is similar to #286 which was fixed for ProtobufCodec in PR #389, but the same fix was not applied to AvroSchemaCodec.

Reproduction

Minimal example (Scala 3):

import zio.schema.{DeriveSchema, Schema}
import zio.schema.codec.AvroSchemaCodec

// Recursive sealed trait: Container holds List[Recursive] where Recursive includes Container
sealed trait Recursive
object Recursive:
  case class Container(items: List[Recursive]) extends Recursive
  case class Value(x: Int) extends Recursive
  given schema: Schema[Recursive] = DeriveSchema.gen[Recursive]

// This causes StackOverflowError
AvroSchemaCodec.encodeToApacheAvro(Recursive.schema)

Scala 2 equivalent:

sealed trait Recursive
object Recursive {
  case class Container(items: List[Recursive]) extends Recursive
  case class Value(x: Int) extends Recursive
  implicit val schema: Schema[Recursive] = DeriveSchema.gen[Recursive]
}

Stack Trace

java.lang.StackOverflowError
  at zio.schema.Schema$Enum2.cases(Schema.scala:1024)
  at zio.schema.codec.AvroSchemaCodec$.toAvroEnum(AvroSchemaCodec.scala:547)
  at zio.schema.codec.AvroSchemaCodec$.toAvroSchema(AvroSchemaCodec.scala:280)
  at zio.schema.codec.AvroSchemaCodec$.toAvroSchema(AvroSchemaCodec.scala:284)
  at zio.schema.codec.AvroSchemaCodec$.toAvroRecordField(AvroSchemaCodec.scala:673)
  at zio.schema.codec.AvroSchemaCodec$.extractAvroFields$$anonfun$1(AvroSchemaCodec.scala:594)
  at zio.Chunk.map(Chunk.scala:44)
  at zio.schema.codec.AvroSchemaCodec$.extractAvroFields(AvroSchemaCodec.scala:594)
  at zio.schema.codec.AvroSchemaCodec$.toAvroRecord$$anonfun$1$$anonfun$1(AvroSchemaCodec.scala:612)
  at zio.schema.codec.AvroSchemaCodec$.toAvroRecord(AvroSchemaCodec.scala:604)
  at zio.schema.codec.AvroSchemaCodec$.toAvroSchema(AvroSchemaCodec.scala:281)
  at zio.schema.codec.AvroSchemaCodec$.$anonfun$27(AvroSchemaCodec.scala:577)
  at zio.schema.codec.AvroSchemaCodec$.toAvroEnum(AvroSchemaCodec.scala:566)
  ... (cycle repeats)

The cycle: toAvroEnumtoAvroSchematoAvroRecordextractAvroFieldstoAvroRecordFieldtoAvroSchematoAvroEnum

Expected Behavior

The codec should detect the cycle and either:

  1. Use Avro's named type reference mechanism (Avro supports recursive schemas via type name references)
  2. Return a Left with an appropriate error message indicating recursive types are not supported

Root Cause

The call chain has no cycle detection. When processing a sealed trait that contains a case class with a field of type List[SealedTrait], it recursively processes the element type, which leads back to the same sealed trait indefinitely.

Suggested Fix

Add cycle detection similar to what was done in PR #389 for ProtobufCodec. Track "in-flight" type names during schema generation and either:

  • Return a named reference when encountering an in-flight type (Avro supports this)
  • Use a memoization/caching approach to avoid re-processing completed types

Example approach:

private def toAvroSchema(
  schema: Schema[_],
  inFlight: Set[String] = Set.empty
): Either[String, org.apache.avro.Schema] = {
  val typeName = getTypeName(schema)
  if (inFlight.contains(typeName)) {
    // Return named reference for recursive type
    Right(org.apache.avro.Schema.createRecord(typeName, null, namespace, false))
  } else {
    // Process with updated inFlight set
    processSchema(schema, inFlight + typeName)
  }
}

Environment

  • zio-schema version: 1.7.6 (latest as of Jan 2026)
  • Scala version: 3.3.4
  • JDK: 21

Confirmed Test Results

Tested with zio-schema 1.7.6 - all three recursive patterns fail:

Pattern Result
List[SealedTrait] StackOverflowError
Option[SealedTrait] StackOverflowError
Direct field reference StackOverflowError

Full reproduction project:

// build.sbt
ThisBuild / scalaVersion := "3.3.4"
libraryDependencies ++= Seq(
  "dev.zio" %% "zio-schema" % "1.7.6",
  "dev.zio" %% "zio-schema-avro" % "1.7.6",
  "dev.zio" %% "zio-schema-derivation" % "1.7.6"
)

// src/main/scala/Repro.scala
import zio.schema.{DeriveSchema, Schema}
import zio.schema.codec.AvroSchemaCodec

sealed trait Recursive
object Recursive:
  case class Container(items: List[Recursive]) extends Recursive
  case class Value(x: Int) extends Recursive
  given schema: Schema[Recursive] = DeriveSchema.gen[Recursive]

@main def run(): Unit =
  println(AvroSchemaCodec.encodeToApacheAvro(Recursive.schema))
  // Throws StackOverflowError

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions