Description
The chunk encoder used by the decision log plugin has these issues:
1. Checking if the incoming event is too large compared the uncompressed size to the compressed configurable limit which could cause an incoming event to lose the ND cache unnecessarily or be dropped entirely. The event could potentially fit if it had been compressed. Comparing it to the "uncompressed limit" (aka the soft limit) isn't possible because it could mean the limit hasn't grown big enough. You can see this happen in TestChunkMaxUploadSizeLimitNDBCacheDropping, when compressed the event could fit within the upload limit of 400 bytes if it was compressed.
2. The size and event buffer reset the chunk encoder on each upload, discarding the soft limit which requires multiple decision events to stabilize. event buffer: https://github.com/open-policy-agent/opa/blob/main/v1/plugins/logs/eventBuffer.go#L131 and size buffer: https://github.com/open-policy-agent/opa/blob/main/v1/plugins/logs/plugin.go#L906-L915
3. The term "soft limit" was confusing, see my previous PR where I went on a journey to find the meaning of soft limit haha #7517
4. The configurable decision_logs.reporting.upload_size_limit_bytes
is used to calculate the uncompressed size by exponentially increasing it, this could cause an overflow if it was set too high. https://github.com/open-policy-agent/opa/blob/main/v1/plugins/logs/encoder.go#L144-L153
Metadata
Metadata
Assignees
Type
Projects
Status