Skip to content

record4s-circe encoder produces non-deterministic JSON field order #169

@haskiindahouse

Description

@haskiindahouse

The circe Codec for record4s collects fields into a Map.newBuilder[String, Json]. The default Map builder produces a hash-ordered Map, so the resulting JSON has fields in hash order rather than the declaration order of the record's labels. For records with more than four fields the order changes silently from run to run (or between Scala versions, since 2.13 vs 3 use different default Map implementations for small sizes).

Source:

private inline def encodeFields[Types, Labels](
record: Map[String, Any],
res: Builder[(String, Json), Map[String, Json]] =
Map.newBuilder[String, Json],
): Builder[(String, Json), Map[String, Json]] =
inline (erasedValue[Types], erasedValue[Labels]) match {
case _: (EmptyTuple, EmptyTuple) =>
res
case _: (tpe *: types, label *: labels) =>
val labelStr = constValue[label & String]
val value =
inline erasedValue[tpe] match {
case _: Json =>
record(labelStr).asInstanceOf[Json]
case _ =>
val enc = summonInline[circe.Encoder[tpe]]
enc(record(labelStr).asInstanceOf[tpe])
}
res += (labelStr -> value)
encodeFields[types, labels](record, res)
}
private inline def decodeFields[Types, Labels](
c: HCursor,
res: Builder[(String, Any), Map[String, Any]] = Map.newBuilder[String, Any],

private inline def encodeFields[Types, Labels](
    record: Iterable[(String, Any)],
    res: Builder[(String, Json), Map[String, Json]] =
      Map.newBuilder[String, Json],
): Map[String, Json] =
  // ...
  res: Builder[(String, Any), Map[String, Any]] = Map.newBuilder[String, Any],

Map.newBuilder is the issue — switching to scala.collection.immutable.ListMap.newBuilder (or LinkedHashMap if mutability is fine inside the helper) preserves insertion order, which is the order labels are visited via the Labels tuple. The record4s core API already promises that record fields are observed in declaration order in iterableOf, so the encoder is the only place dropping that guarantee.

The behaviour matters for: snapshot tests against expected JSON strings, hashing/signing JSON payloads, and any consumer that relies on field order (e.g. canonical JSON, JWT-claim ordering).

Reproducer (scala-cli):

//> using scala 3.8.3
//> using dep "com.github.tarao::record4s::0.13.0"
//> using dep "com.github.tarao::record4s-circe::0.13.0"
//> using dep "io.circe::circe-core::0.14.10"

import com.github.tarao.record4s.%
import com.github.tarao.record4s.circe.Codec.given
import io.circe.syntax.*

@main def repro(): Unit =
  val r = %(a = 1, b = 2, c = 3, d = 4, e = 5, f = 6, g = 7)
  println(r.asJson.spaces2)

The output won't match the declaration order a, b, c, d, e, f, g — instead you get whatever order HashMap happens to use for those keys at that size.

A drop-in fix is scala.collection.immutable.ListMap.newBuilder[String, Json] plus the matching change on the Map[String, Any] builder a few lines down. Happy to PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions