-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Labels
Description
We do a lot of unnecessary copying and cloning of dictionary-encoded structs currently, especially during encoding.
Consider the following:
- Allow marking structs frozen. Once frozen they can be safely referenced by pointer from multiple places (including dictionaries) without cloning.
- Use freezing after record is read.
- Allowing referencing frozen structs from read operations directly in write operations. Should improve copying (read/write).
- Make CopyFrom smarter, avoid cloning frozen structs, just copy the pointer.
- Encourage Freeze() when creating data (not from Read()) of Writer. Should improve encoding.
- Cache refNum in the object to avoid slow Tree lookups.
- Replace Tree by Hashmap. Cache the hashcode, invalidate on modifications.
- Allowing setting dictionary structs in their containing structs by pointer instead of requiring full CopyFrom or field-by-field setting.
Most of these can be applied to varying degrees both to Go and Java. This summary issue is for Go. Duplicate it for Java.
See some initial experiment on this branch https://github.com/splunk/stef/compare/main...tigran/profileserialize?expand=1