Error: A code error has led to negative space remaining

[ClaymorePlay]

I am doing multiple line insertion into a file from BD but sometimes I get an error like this, I would like to know what it means

ArithmeticException: A code error has led to negative space remaining at ApacheOrcDotNet.Compression.OrcCompressedBuffer.Write(Byte[] buffer, Int32 offset, Int32 count) at ApacheOrcDotNet.ColumnTypes.StringWriter.AddBlock(IList1 values) at ApacheOrcDotNet.Stripes.StripeWriter.<>c__DisplayClass33_01.b__1() at ApacheOrcDotNet.Stripes.StripeWriter.CompleteStride() at ApacheOrcDotNet.Stripes.StripeWriter.RowAddingCompleted() at ApacheOrcDotNet.OrcWriter`1.Dispose() at Workers.ArchivingWorker.Jobs.ArchivingEndpointRequestFull.ExecuteAsync(IServiceProvider serviceProvider) in /src/Workers.ArchivingWorker/Jobs/ArchivingEndpointRequestFull.cs:line 187

[ddrinka]

Hi @ClaymorePlay, it looks like if a single string is longer than WriterConfiguration.BufferSize, which by default is 256KB, this exception would be thrown. Without writing a test to confirm, I'm also suspicious that writing two strings that add up to more than 256KB would trigger this exception.

Are you writing long strings?

[ClaymorePlay]

Yes, the recording takes place with an unpredictable size, I’ll try to increase the buffer size

[ClaymorePlay]

Increasing the buffer size helped, thank you. I have now set it to 1GB, the process has started

[ClaymorePlay]

Next error: the author needs to increase the size of more than just the buffer.

OverflowException: Compressed block size cannot be larger than 8*3-1 bits (8MB). Attempted to store 44736697 bytes. at ApacheOrcDotNet.Compression.OrcCompressedStream.WriteBlockHeader(Stream outputStream, Int32 blockLength, Boolean isCompressed) at ApacheOrcDotNet.Compression.OrcCompressedStream.CompressCopyTo(MemoryStream uncompressedSource, Stream compressedDestination, CompressionKind compressionKind, CompressionStrategy compressionStrategy) at ApacheOrcDotNet.Compression.OrcCompressedBuffer.Flush() at ApacheOrcDotNet.ColumnTypes.StringWriter.FlushBuffers() at ApacheOrcDotNet.Stripes.StripeWriter.CompleteStripe() at ApacheOrcDotNet.Stripes.StripeWriter.RowAddingCompleted() at ApacheOrcDotNet.OrcWriter`1.Dispose() at Workers.ArchivingWorker.Jobs.ArchivingEndpointRequestFull.ExecuteAsync(IServiceProvider serviceProvider) in /src/Workers.ArchivingWorker/Jobs/ArchivingEndpointRequestFull.cs:line 198

[ClaymorePlay]

Please tell me the normal configuration in case of unknown file size. If you specify BlockSize instead of BufferSize, the problem appears like this:

ArithmeticException: A code error has led to negative space remaining at ApacheOrcDotNet.Compression.OrcCompressedBuffer.Write(Byte[] buffer, Int32 offset, Int32 count) at ApacheOrcDotNet.ColumnTypes.StringWriter.AddBlock(IList1 values) at ApacheOrcDotNet.Stripes.StripeWriter.<>c__DisplayClass33_01.b__1() at ApacheOrcDotNet.Stripes.StripeWriter.CompleteStride() at ApacheOrcDotNet.Stripes.StripeWriter.RowAddingCompleted() at ApacheOrcDotNet.OrcWriter`1.Dispose() at Workers.ArchivingWorker.Jobs.ArchivingEndpointRequestFull.ExecuteAsync(IServiceProvider serviceProvider) in /src/Workers.ArchivingWorker/Jobs/ArchivingEndpointRequestFull.cs:line 198

[ClaymorePlay]

What happens if you ignore the error regarding 8MB? I did this, the file is written, but is it in the correct format?

[ClaymorePlay]

The problem disappeared when I returned Write Configuration to its default state again without setting values...

[ClaymorePlay]

The original problem is still there, waiting for a response. ArithmeticException: A code error has led to negative space remaining at ApacheOrcDotNet.Compression.OrcCompressedBuffer.Write(Byte[] buffer, Int32 offset, Int32 count) at ApacheOrcDotNet.ColumnTypes.StringWriter.AddBlock(IList1 values) at ApacheOrcDotNet.Stripes.StripeWriter.<>c__DisplayClass33_01.b__1() at ApacheOrcDotNet.Stripes.StripeWriter.CompleteStride() at ApacheOrcDotNet.Stripes.StripeWriter.RowAddingCompleted() at ApacheOrcDotNet.OrcWriter`1.Dispose() at Workers.ArchivingWorker.Jobs.ArchivingEndpointRequestFull.ExecuteAsync(IServiceProvider serviceProvider) in /src/Workers.ArchivingWorker/Jobs/ArchivingEndpointRequestFull.cs:line 187