Skip to content

[BUG]: Wrong stats for Timestamp micros #769

Description

@HaroldJaya

Library Version

6.0.3

.NET Runtime

.Net 10

OS and Architecture

Windows (64-bit)

How to reproduce?

Run the test.
The min and max are encoded without considering the Unit, saw some improvements over units are expected for next versions, thanks!

Failing test

[Fact]
public async Task InvalidStatisticsForMicrosDateTime() {
    var ms = new MemoryStream();
    const long micros = 1767225600123456;
    var element = new TestDateStats {
        Date = DateTimeOffset.UnixEpoch.AddTicks(micros * 10).DateTime
    };
    var input = new TestDateStats[] { element };

    await ParquetSerializer.SerializeAsync(input, ms);

    // read back
    ms.Position = 0;
    await using(ParquetReader r = await ParquetReader.CreateAsync(ms)) {
        using ParquetRowGroupReader rgr = r.OpenRowGroupReader(0);
        DataColumnStatistics? stats = rgr.GetStatistics(r.Schema.DataFields.Single(field => field.Name == "Date"));
        long max = (long)stats.MaxValue;
        long min = (long)stats.MinValue;
        Assert.Equal(micros, max);
        Assert.Equal(micros, min);
    }
}

public class TestDateStats {
    [ParquetTimestamp(ParquetTimestampResolution.Microseconds, true)]
    public DateTime Date { get; set; }
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions