Skip to content

Conversation

Copy link

Copilot AI commented Aug 11, 2025

This PR updates the Compressor interface and all its implementations to return the length of the encoded buffer in addition to the buffer itself and any error. The Encode() method signature changes from ([]byte, error) to ([]byte, int, error).

Changes Made

Interface Update

  • Modified the Compressor interface in compressor.go to include length in the return signature
  • Updated both SnappyCompressor and LZ4Compressor implementations

Implementation Details

  • SnappyCompressor: Now returns (encoded, len(encoded), nil) where the length matches the actual encoded buffer size
  • LZ4Compressor: Now returns (encoded, len(encoded), nil) where the length includes the 4-byte Cassandra length prefix
  • Frame handling: Updated frame.go to use the returned length when setting compressed frame length instead of recalculating it

Test Updates

  • Updated all tests in compressor_test.go and lz4_test.go to handle the new 3-parameter return
  • Tests use _ to ignore the length parameter when not needed for validation, avoiding redundant len() calls that would defeat the optimization purpose
  • All existing functionality preserved with no breaking changes to compression/decompression behavior

Validation

The changes maintain full backward compatibility in terms of functionality while providing the requested length information. All unit tests pass, and both Snappy and LZ4 compression continue to work as expected.

Example usage:

compressor := gocql.SnappyCompressor{}
encoded, length, err := compressor.Encode(data)
if err != nil {
    return err
}
// length now contains the size of the encoded buffer
fmt.Printf("Compressed %d bytes to %d bytes\n", len(data), length)

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@mykaul
Copy link

mykaul commented Aug 11, 2025

Run @copilot , Run! (I hope you understand the reference to Forrest Gump)

Copilot AI changed the title [WIP] Update Encode() functions to return encoded length, change interface and usages Update Encode() interface to return buffer length for compressors Aug 11, 2025
Copilot finished work on behalf of mykaul August 11, 2025 17:47
Copilot AI requested a review from mykaul August 11, 2025 17:47
Copilot finished work on behalf of mykaul August 11, 2025 17:52
@dkropachev
Copy link
Collaborator

I don't think we need that.

@mykaul
Copy link

mykaul commented Aug 11, 2025

I don't think we need that.

It actually started with me chatting with CoPi and an experiment what can it do for me. Haven't looked at its code yet.

Nevertheless, I don't see why we wouldn't want it, and you haven't provided a reason.
In every language, I'd skip a call of len(), even if I suspect it's an o(1) here, for the objects being called. (In Python it usually is, unsure about Golang), and even though there's a good chance it is optimized by the compiler here. If the code isn't less readable / complex, I'm not sure why not.

@dkropachev
Copy link
Collaborator

dkropachev commented Aug 11, 2025

I don't think we need that.

It actually started with me chatting with CoPi and an experiment what can it do for me. Haven't looked at its code yet.

Nevertheless, I don't see why we wouldn't want it, and you haven't provided a reason. In every language, I'd skip a call of len(), even if I suspect it's an o(1) here, for the objects being called. (In Python it usually is, unsure about Golang), and even though there's a good chance it is optimized by the compiler here. If the code isn't less readable / complex, I'm not sure why not.

Two reasons:

  1. It does not give significant performance boost while complicating code
  2. This kind of optimizations should be on the side of the compiler
  3. It is orthogonal to the place where this API should go - streaming API

t.Fatalf("failed to encode '%v' with error %v", str, err)
} else if bytes.Compare(expected, res) != 0 {
t.Fatal("failed to match the expected encoded value with the result encoded value.")
} else if length != len(res) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot - what's the point of this additional check? We are trying to reduce the no. of len() calls here, not re-add them.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right. I removed the redundant len() calls from the tests since they defeat the purpose of the optimization. The tests now use _ to ignore the length parameter when it's not needed for validation. Changes committed in c9b5a7e.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see commit c9b5a7e - perhaps because this PR is closed.

@mykaul
Copy link

mykaul commented Aug 12, 2025

I don't think we need that.

It actually started with me chatting with CoPi and an experiment what can it do for me. Haven't looked at its code yet.
Nevertheless, I don't see why we wouldn't want it, and you haven't provided a reason. In every language, I'd skip a call of len(), even if I suspect it's an o(1) here, for the objects being called. (In Python it usually is, unsure about Golang), and even though there's a good chance it is optimized by the compiler here. If the code isn't less readable / complex, I'm not sure why not.

Two reasons:

  1. It does not give significant performance boost while complicating code

Indeed it doesn't give a performance boost. Probably micro one at best. It doesn't complicate the code though, as I've shown in the Python driver. The changes are minimal and readable.

  1. This kind of optimizations should be on the side of the compiler

Yes, but the compiler can do a better job with good code than mediocre code.

  1. It is orthogonal to the place where this API should go - streaming API

I'm not sure what that means.

Nevertheless, closing the PR.

@mykaul mykaul closed this Aug 12, 2025
Copilot AI requested a review from mykaul August 12, 2025 06:12
Copilot finished work on behalf of mykaul August 12, 2025 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants