Skip to content

FEAT: spec compliannce - allow reading non compressed flatbuffers #1988

@ianhi

Description

@ianhi

Per spec flatbuffers have either ZStd or None compression.

/// Compression algorithm used for metadata files.
#[repr(u8)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CompressionAlgorithmBin {
None = 0u8,
Zstd = 1u8,
}

but currently we only support writing and reading. zstd

async fn check_and_decompress(
mut read: Pin<Box<dyn AsyncBufRead + Send>>,
file_type: FileTypeBin,
) -> RepositoryResult<(SpecVersionBin, Vec<u8>)> {
let (spec_version, compression) = check_header(&mut read, file_type).await?;
debug_assert_eq!(compression, CompressionAlgorithmBin::Zstd);
let mut decoder = ZstdDecoder::new(read);
let mut buffer = Vec::new();
decoder.read_to_end(&mut buffer).await.capture()?;
buffer.shrink_to_fit();
Ok((spec_version, buffer))

at a minimum we need to support reading

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions