-
Notifications
You must be signed in to change notification settings - Fork 194
Description
Description
AvroSchemaCodec.encodeToApacheAvro causes a StackOverflowError when encoding a schema that contains a recursive sealed trait pattern - specifically when a case class within a sealed trait contains a List[SealedTrait] (or similar collection) referring back to the parent sealed trait.
This is similar to #286 which was fixed for ProtobufCodec in PR #389, but the same fix was not applied to AvroSchemaCodec.
Reproduction
Minimal example (Scala 3):
import zio.schema.{DeriveSchema, Schema}
import zio.schema.codec.AvroSchemaCodec
// Recursive sealed trait: Container holds List[Recursive] where Recursive includes Container
sealed trait Recursive
object Recursive:
case class Container(items: List[Recursive]) extends Recursive
case class Value(x: Int) extends Recursive
given schema: Schema[Recursive] = DeriveSchema.gen[Recursive]
// This causes StackOverflowError
AvroSchemaCodec.encodeToApacheAvro(Recursive.schema)Scala 2 equivalent:
sealed trait Recursive
object Recursive {
case class Container(items: List[Recursive]) extends Recursive
case class Value(x: Int) extends Recursive
implicit val schema: Schema[Recursive] = DeriveSchema.gen[Recursive]
}Stack Trace
java.lang.StackOverflowError
at zio.schema.Schema$Enum2.cases(Schema.scala:1024)
at zio.schema.codec.AvroSchemaCodec$.toAvroEnum(AvroSchemaCodec.scala:547)
at zio.schema.codec.AvroSchemaCodec$.toAvroSchema(AvroSchemaCodec.scala:280)
at zio.schema.codec.AvroSchemaCodec$.toAvroSchema(AvroSchemaCodec.scala:284)
at zio.schema.codec.AvroSchemaCodec$.toAvroRecordField(AvroSchemaCodec.scala:673)
at zio.schema.codec.AvroSchemaCodec$.extractAvroFields$$anonfun$1(AvroSchemaCodec.scala:594)
at zio.Chunk.map(Chunk.scala:44)
at zio.schema.codec.AvroSchemaCodec$.extractAvroFields(AvroSchemaCodec.scala:594)
at zio.schema.codec.AvroSchemaCodec$.toAvroRecord$$anonfun$1$$anonfun$1(AvroSchemaCodec.scala:612)
at zio.schema.codec.AvroSchemaCodec$.toAvroRecord(AvroSchemaCodec.scala:604)
at zio.schema.codec.AvroSchemaCodec$.toAvroSchema(AvroSchemaCodec.scala:281)
at zio.schema.codec.AvroSchemaCodec$.$anonfun$27(AvroSchemaCodec.scala:577)
at zio.schema.codec.AvroSchemaCodec$.toAvroEnum(AvroSchemaCodec.scala:566)
... (cycle repeats)
The cycle: toAvroEnum → toAvroSchema → toAvroRecord → extractAvroFields → toAvroRecordField → toAvroSchema → toAvroEnum
Expected Behavior
The codec should detect the cycle and either:
- Use Avro's named type reference mechanism (Avro supports recursive schemas via type name references)
- Return a
Leftwith an appropriate error message indicating recursive types are not supported
Root Cause
The call chain has no cycle detection. When processing a sealed trait that contains a case class with a field of type List[SealedTrait], it recursively processes the element type, which leads back to the same sealed trait indefinitely.
Suggested Fix
Add cycle detection similar to what was done in PR #389 for ProtobufCodec. Track "in-flight" type names during schema generation and either:
- Return a named reference when encountering an in-flight type (Avro supports this)
- Use a memoization/caching approach to avoid re-processing completed types
Example approach:
private def toAvroSchema(
schema: Schema[_],
inFlight: Set[String] = Set.empty
): Either[String, org.apache.avro.Schema] = {
val typeName = getTypeName(schema)
if (inFlight.contains(typeName)) {
// Return named reference for recursive type
Right(org.apache.avro.Schema.createRecord(typeName, null, namespace, false))
} else {
// Process with updated inFlight set
processSchema(schema, inFlight + typeName)
}
}Environment
- zio-schema version: 1.7.6 (latest as of Jan 2026)
- Scala version: 3.3.4
- JDK: 21
Confirmed Test Results
Tested with zio-schema 1.7.6 - all three recursive patterns fail:
| Pattern | Result |
|---|---|
List[SealedTrait] |
StackOverflowError |
Option[SealedTrait] |
StackOverflowError |
| Direct field reference | StackOverflowError |
Full reproduction project:
// build.sbt
ThisBuild / scalaVersion := "3.3.4"
libraryDependencies ++= Seq(
"dev.zio" %% "zio-schema" % "1.7.6",
"dev.zio" %% "zio-schema-avro" % "1.7.6",
"dev.zio" %% "zio-schema-derivation" % "1.7.6"
)
// src/main/scala/Repro.scala
import zio.schema.{DeriveSchema, Schema}
import zio.schema.codec.AvroSchemaCodec
sealed trait Recursive
object Recursive:
case class Container(items: List[Recursive]) extends Recursive
case class Value(x: Int) extends Recursive
given schema: Schema[Recursive] = DeriveSchema.gen[Recursive]
@main def run(): Unit =
println(AvroSchemaCodec.encodeToApacheAvro(Recursive.schema))
// Throws StackOverflowErrorRelated Issues
- Recursive data-type can't be encoded with
ProtobufCodec#286 - Recursive data-type can't be encoded with ProtobufCodec (FIXED in PR Make DynamicValue.fromSchemaAndValue and codecs stack safe #389) - java.lang.StackOverflowError on DeriveSchema.gen #159 - java.lang.StackOverflowError on DeriveSchema.gen (different issue - macro-level)