Skip to content

serialization-cbor.md

Latest commit

 

History

History
287 lines (206 loc) · 14.7 KB

serialization-cbor.md

File metadata and controls

287 lines (206 loc) · 14.7 KB
Raw

Concise Binary Object Representation (CBOR) is a compact binary format based on JSON. It supports a subset of JSON features and produces binary output instead of text.

Add dependencies for CBOR

To use CBOR in your project, add the CBOR serialization library dependency to your build file:

// build.gradle(.kts)

dependencies {
    implementation("org.jetbrains.kotlinx:kotlinx-serialization-cbor:%serializationVersion%")
}
<!-- pom.xml -->

<dependencies>
    <dependency>
        <groupId>org.jetbrains.kotlinx</groupId>
        <artifactId>kotlinx-serialization-cbor</artifactId>
        <version>%serializationVersion%</version>
    </dependency>
</dependencies>

Use CBOR for binary serialization

The Cbor class provides two main functions:

Let's look at an example where a Project object is serialized into a binary array and then deserialized back to its original form:

// Imports declarations from the serialization library
import kotlinx.serialization.*
import kotlinx.serialization.cbor.*

fun ByteArray.toAsciiHexString() = joinToString("") {
    // Shows printable ASCII bytes as characters and other bytes as hex values
    if (it in 32..127) it.toInt().toChar().toString() else
        "{${it.toUByte().toString(16).padStart(2, '0').uppercase()}}"
}

@Serializable
data class Project(val name: String, val language: String)

@OptIn(ExperimentalSerializationApi::class)
fun main() {
    val data = Project("kotlinx.serialization", "Kotlin")
    
    // Serializes the object to a CBOR binary array
    val bytes = Cbor.encodeToByteArray(data)

    // Converts the binary array to a human-readable hex string
    println(bytes.toAsciiHexString())
    // {BF}dnameukotlinx.serializationhlanguagefKotlin{FF}
    
    // Deserializes the binary array back to an object
    val obj = Cbor.decodeFromByteArray<Project>(bytes)
    println(obj)
    // Project(name=kotlinx.serialization, language=Kotlin)
}

This example prints the encoded bytes in a readable mixed form. It represents printable ASCII bytes as characters and non-printable bytes as hexadecimal values.

The following table shows the same output in full CBOR hex notation:

Hex Code CBOR Type Description
BF map(*) Start of a CBOR map
64 text(4) Length of the string
6E616D65 string The string "name"
75 text(21) Length of the string
6B6F746C696E782E73657269616C697A6174696F6E string The string "kotlinx.serialization"
68 text(8) Length of the string
6C616E6775616765 string The string "language"
66 text(6) Length of the string
4B6F746C696E string The string "Kotlin"
FF primitive(*) End of the CBOR map

Ignore unknown keys in CBOR

CBOR is commonly used in communication with IoT devices where new properties may be added as part of API evolution. By default, unknown keys encountered during deserialization result in an error.

Just like in JSON, you set the ignoreUnknownKeys property to true to ignore them during deserialization:

// Imports declarations from the serialization library
import kotlinx.serialization.*
import kotlinx.serialization.cbor.*

@Serializable
data class Project(val name: String)

@OptIn(ExperimentalSerializationApi::class)
fun main() {

    // Creates a Cbor instance that ignores unknown keys during deserialization
    val format = Cbor { ignoreUnknownKeys = true }

    // Decodes the CBOR input and ignores the unknown "language" key
    val data = format.decodeFromHexString<Project>(
        // CBOR hex notation input with an extra, unknown "language" key
        "bf646e616d65756b6f746c696e782e73657269616c697a6174696f6e686c616e6775616765664b6f746c696eff"
    )
    println(data)
    // Project(name=kotlinx.serialization)
}

In this CBOR input, the following bytes represent the unknown "language" key:

  • 68: Length of the key "language"
  • 6c616e6775616765: The key "language"
  • 66: Length of the value "Kotlin"
  • 4b6f746c696e: The value "Kotlin"

Customize how CBOR encodes data

According to the RFC 8949 Major Types specification, CBOR supports the following data types:

  • Major type 0: an unsigned integer
  • Major type 1: a negative integer
  • Major type 2: a byte string
  • Major type 3: a text string
  • Major type 4: an array of data items
  • Major type 5: a map of pairs of data items
  • Major type 6: optional semantic tagging of other major types
  • Major type 7: floating-point numbers, simple data types with no content, and the "break" stop code

By default, Kotlin ByteArray instances are encoded as major type 4, which represents an array of data items.

To encode ByteArray instances as major type 2, a byte string, use the @ByteString annotation:

// Imports declarations from the serialization library
import kotlinx.serialization.*
import kotlinx.serialization.cbor.*

fun ByteArray.toAsciiHexString() = joinToString("") {
    if (it in 32..127) it.toInt().toChar().toString() else
        "{${it.toUByte().toString(16).padStart(2, '0').uppercase()}}"
}

@Serializable
data class Data(
    // Encodes the byte array as CBOR major type 2 as a byte string
    @ByteString
    val type2: ByteArray,
    // Encodes the byte array as CBOR major type 4 as an array of individual data items
    val type4: ByteArray
)

@OptIn(ExperimentalSerializationApi::class)
fun main() {
    // Creates a Data object with two ByteArray fields
    val data = Data(byteArrayOf(1, 2, 3, 4), byteArrayOf(5, 6, 7, 8))

    // Serializes the Data object into a CBOR byte array
    val bytes = Cbor.encodeToByteArray(data)

    println(bytes.toAsciiHexString())
    // {BF}etype2D{01}{02}{03}{04}etype4{9F}{05}{06}{07}{08}{FF}{FF}
    
    val obj = Cbor.decodeFromByteArray<Data>(bytes)
    println(obj)
    // Data(type2=[1, 2, 3, 4], type4=[5, 6, 7, 8])
}

In this example, the bytes before each ByteArray value differ because the properties use different CBOR major types.

Instead of annotating each property with @ByteString, you can also encode all ByteArray values as major type 2 by setting the alwaysUseByteString property to true.

{style="note"}

You can also customize how CBOR encodes entire classes.

By default, classes are serialized as a CBOR map, which corresponds to major type 5. This means that each property of the class is stored as a key-value pair.

You can serialize a class as a CBOR array, major type 4, with the @CborArray annotation. This can be useful for encoding COSE message structures, which RFC 9052 defines as CBOR arrays.

Here's an example:

@Serializable
@CborArray
data class DataClass(
    val alg: Int,
    val kid: String?
)
 
Cbor.encodeToByteArray(DataClass(alg = -7, kid = null))

With the @CborArray annotation, this example is encoded as a CBOR array: 0x8226f6. Without it, the same class is encoded as a CBOR map: 0xa263616c6726636b6964f6.

Unlike JSON, CBOR supports maps with non-trivial keys. Some parsers, such as jackson-dataformat-cbor, don't support this feature.

For a JSON workaround, see Allow structured map keys.

{style="tip"}

By using annotations like @ByteString and @CborArray, you can customize how CBOR encodes data to better match existing specifications and, in some cases, reduce binary size.

Definite and indefinite length encoding in CBOR

CBOR supports two encodings for maps and arrays: definite length encoding and indefinite length encoding.

By default, Kotlin serialization uses indefinite length encoding. This means that the number of elements in a map or array isn't encoded explicitly, and a terminating byte is appended after the last element.

Definite length encoding omits the terminating byte and encodes the number of elements at the start of the map or array.

To switch between these two modes, use the useDefiniteLengthEncoding property.

Tags and labels in CBOR

CBOR allows you to define tags that encode additional information for properties and values. You can specify these tags with the @KeyTags and @ValueTags annotations. The encodeKeyTags, encodeValueTags, verifyKeyTags, and verifyValueTags properties control the encoding and verification of these tags.

For more information on tagging in CBOR, see RFC 8949 Tagging of Items.

{style="tip"}

You can also tag classes using the @ObjectTags annotation, which applies tags to all instances of a class.

When serializing, @ObjectTags are encoded directly before the data of the tagged object. If a property has value tags and its type has object tags, the value tags are encoded before the object tags. The encodeObjectTags and verifyObjectTags properties control whether object tags are encoded and verified. If you verify only value tags and don't verify object tags, the decoder can still deserialize data with additional object tags.

For a list of well-known tags, see CborTag.

{style="tip"}

CBOR supports map keys of any type. In COSE (CBOR Object Signing and Encryption), these keys are restricted to strings and numbers and are called labels.

You can assign string labels with the @SerialName annotation and numeric labels with the @CborLabel annotation. The preferCborLabelsOverNames property allows prioritizing numeric labels over serial names when both are present. You can use it to keep compact labels for CBOR while still keeping readable names when serializing to JSON.

Kotlin serialization also provides a predefined Cbor.CoseCompliant instance that follows COSE encoding requirements. It uses definite length encoding, encodes and verifies all tags, and prefers numeric labels over serial names.

Custom CBOR-specific serializers

CBOR encoders and decoders implement the CborEncoder and CborDecoder interfaces.

These interfaces extend the general Encoder and Decoder APIs, providing access to CBOR-specific configurations through the cbor property. Custom serializers can use this property to access the current Cbor instance, produce embedded byte arrays, and read the current settings, such as preferCborLabelsOverNames and useDefiniteLengthEncoding.

For more information about creating custom serializers, see Create custom serializers.