Possible Compression Improvements

[william-silversmith]

• Experiment more with pins
• Relative coding of chains to increase letter frequency
• Code beginning of chain (BOC) using rows
• Reduce BOC markers by allowing a different color pen to connect connected components
• Allow tracing around the edge of the image if it would eliminate enough BOC
• 3D motion prediction for crack codes (hard!)
• Hexagonal pixels? (1 bit per a direction change, similar path length, https://art-of-logic.blogspot.com/2014/12/hexagonal-grid-city-life.html)

[william-silversmith]

I tried the BOC optimization, but the image statistics are such that not too many chains start from the same row and so the overheads made it so that the optimized version was ~0.5% bigger. Maybe this could be optimized by picking the starts to be along a row.

You run into the set cover problem again. :P

[william-silversmith]

Managed to improve pins such that on connectomics.npy, I am seeing a 4x reduction in the amount of data pre-gzip. After gzip? A 7% improvement. Pins are already slow, but a better optimizer than greedy might result in further substantial gains (at high compute cost).

[william-silversmith]

BOC are pretty small... ~2% in connectomics.npy. Marginal improvements possible even though there may be substantial gains on that 2%.