Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support split encode/decoder mode, "mode B" #27

Merged
merged 33 commits into from
Feb 26, 2024
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
bcae6c1
Possibly interesting grid configurations
sz3 Jun 11, 2023
120f62d
Fix a bug
sz3 Jun 18, 2023
a8cb59c
temp checkpoint? Recap of early 2021 ccm experiments
sz3 Jun 18, 2023
daa828f
Optionally use kmeans clustering to find "target" colors?
sz3 Jun 19, 2023
a554c9d
Bring back relative color diff?
sz3 Jul 2, 2023
e0ce0ee
Change 5x6alt default ecc to 35...
sz3 Jul 2, 2023
65b7868
Switching to a two-pass, symbols *then* colors approach to match libc…
sz3 Jul 2, 2023
6c963dc
Color decode cells by adjacency, not index order
sz3 Jul 2, 2023
3dbeaa8
Match libcimbar's color decode heuristic (for now)
sz3 Oct 21, 2023
e8bf49c
Split symbols and colors for encodes?
sz3 Oct 22, 2023
bbec5d9
Implement split mode for decoder + fix encoder
sz3 Oct 22, 2023
3b1b63d
Cleaning up two-pass decode a bit
sz3 Oct 26, 2023
73445b3
Hold on to fountain headers we see during the decode?
sz3 Oct 26, 2023
e873819
Fix split encode. Fountain blocks now dynamic for newer confs
sz3 Oct 27, 2023
c5863bc
Add SPLIT_MODE toggle to config
sz3 Oct 27, 2023
7236c32
Add some logic to pre-calculate fountain headers for color decode
sz3 Oct 30, 2023
337604c
remove frame_ids from fountain header color decode key... for now?
sz3 Oct 30, 2023
9d139c1
Compute average expected colors before color decode begins
sz3 Nov 6, 2023
9b92357
Update the grader to know how to parse/grade split frames
sz3 Nov 25, 2023
5f4f989
Playing with a global color correction matrix based on avg "key" values
sz3 Dec 3, 2023
d787d48
A few permutations of things to try...
sz3 Dec 3, 2023
35237d6
Bugfix
sz3 Dec 3, 2023
3847fac
Color correction experiments, pt N
sz3 Dec 6, 2023
fa9e21f
Play with adding a two-sector color decode,
sz3 Dec 6, 2023
344bed9
Quick refactor of ccm calculation logic
sz3 Dec 10, 2023
f527646
smash the ccm step together?
sz3 Dec 10, 2023
545bf87
Possible refactor of single-color sample check?
sz3 Dec 26, 2023
e27511d
Docs update
sz3 Feb 16, 2024
e3da330
Update to v0.6 samples, fix tests
sz3 Feb 18, 2024
85b0ddc
Freeze the requirements file for CI
sz3 Feb 19, 2024
452f6d9
Update samples submodule rev + readme sample image
sz3 Feb 22, 2024
ed40321
Update color-correct option
sz3 Feb 22, 2024
bdb35fb
Comments/logs
sz3 Feb 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: [3.7, 3.8, 3.9, "3.10"]
python-version: [3.8, 3.9, "3.10", "3.11"]
steps:
- uses: actions/checkout@v2
with:
Expand All @@ -20,7 +20,7 @@ jobs:

- name: install dependencies
run: |
pip install -r requirements
pip install -r requirements.freeze
pip install --upgrade coveralls

- name: test
Expand Down
11 changes: 5 additions & 6 deletions ABOUT.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ My conclusion was that if I was going to create a proof-of-concept implementatio

Not yet. The project is -- at this point -- just a proof-of-concept. It's not clear to me that cimbar is any better for data density + reliability than HCCB, or better than the myriad of unfinished attempts at color QR codes that are floating around, or (...). It could be a technological dead end. But perhaps with more refinement it might be interesting?

## Notable open questions, concerns, and ideas:
## Unresolved design questions, concerns, and ideas:

* the symbol set is not optimal. There are 16, and their image hashes are all reasonable hamming distance from each other, and do *ok* when upscaled... but I drew the initial 40 or so candidates by hand in kolourpaint, and paired down the set to 16 by experimentation.
* 32 distinct tiles (5 bits) be possible for 8x8 tiles
Expand All @@ -52,21 +52,20 @@ Not yet. The project is -- at this point -- just a proof-of-concept. It's not cl
* 8-color cimbar (3 color bits) is possible, at least in dark mode
* probably light mode as well, but a palette will need to be found
* 16-color cimbar does not seem possible with the current color decoding logic
* but with pre-processing for color correction and a perceptual color diff (like CIE76), ... maybe?
* this may be wishful thinking.
* ...at least not in the small (sub-8x8) tile sizes we want to use for high data density

* Reed Solomon was chosen for error correction due how ubiquitous its implementations are.
* it isn't a perfect fit. Most cimbar errors are 1-3 flipped bits at a time -- Reed Solomon doesn't care if one bit flipped or eight did -- a bad byte is a bad byte.
* Something built on LDPC would likely be better.
* Something that cares about bits and not bytes (LDPC? idk) would likely be better.

* the focus on computer screens has surely overlooked problems that come from paper/printed/e-paper surfaces
* using a black background ("dark mode") came out of getting better results from backlit screens
* notably, there are some threshold parameters and color averaging heuristics that will (probably) need to be re-tuned for "light mode" cimbar
* curved surfaces are a can of worms I didn't want to open -- there are some crazy ideas that could be pursued there, but they may not be worth the effort

* should cimbar include "metadata" to tell the decoder what it's trying to decode -- ECC level, number of colors, (grid size...?)
* the bottom right corner of the image seems like the logical place for this.
* a slightly smaller 4th anchor pattern could give us 11 tiles (44 bits?) to work with for metadata, which is not a lot, but probably enough.
* the bottom right corner of the image seems like the logical place for this. However, differing aspect ratios may be a problem
* example: on 8x8, a slightly smaller 4th anchor pattern could give us 11 tiles (44 bits?) to work with for metadata, which is not a lot, but probably enough to be useful.


## Would you like to know more?
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
cimbar is a proof-of-concept 2D data encoding format -- much like [QR Codes](https://en.wikipedia.org/wiki/QR_code), [JAB codes](https://jabcode.org/), and [Microsoft's HCCB](https://en.wikipedia.org/wiki/High_Capacity_Color_Barcode).

<p align="center">
<img src="https://github.com/sz3/cimbar-samples/blob/v0.5/6bit/4color_ecc30_fountain_0.png" width="70%" title="an example cimbar code" >
<img src="https://github.com/sz3/cimbar-samples/blob/v0.6/b/4cecc30f.png" width="70%" title="A non-animated mode-B cimbar code" >
</p>

## How it works
Expand Down Expand Up @@ -33,16 +33,16 @@ The main constraints cimbar must deal with are:
* all tiles in the tileset must be sufficient hamming distance away from each other, where *sufficient* is determined by whether the decoder can consistently place blurry or otherwise imperfect tiles in the correct "bucket".
* all colors in the colorset must be far enough away from each other -- currently as a function of RGB value scaling -- such that color bleeding, reflections, and the like, can be overcome.

In practice, this means that the source image should be around 900x900 resolution or greater, with reasonable color correction handled by the camera -- as you'd find in any modern cell phone.
Cimbar is designed to deal with some lossiness. In practice, the source image should be around 700x700 resolution or greater, in focus, and with *some* color correction handled by the camera -- as you'll hopefully find in any modern cell phone.

This python cimbar implementation is a research project. It works, but it is not very performant, and does not handle error cases with much grace. [libcimbar](https://github.com/sz3/libcimbar), the C++ implementation, has been much more heavily optimized and tested. The target goals of the proof-of-concept were:
This python cimbar implementation is a research project. It works, but it is slow, and does not handle error cases with much grace. [libcimbar](https://github.com/sz3/libcimbar), the C++ implementation, has been much more heavily optimized and tested. The target goals of the proof-of-concept were:
1. achieve data density on the order of _10kb_ per image.
2. validate a theoretical performance (and if possible, an implemented demonstration) of >= _100kb/s_ data transfer (800 kilobits/second) from a computer screen to a cell phone, using only animated cimbar codes and the cell phone camera.

## I want numbers!

* a 6-bit cimbar image contains `9300` raw bytes of data, and `7500` bytes with the default error correction level (30)
* for 7-bit cimbar, the respective numbers are `10850` and `8750`
* a `mode B` (8x8, 4-color, 30/155 ecc, 6-bits-per-tile) cimbar image contains `9300` raw bytes of data, and `7500` bytes with the default error correction level (30)
* for the old `mode 8C` (8x8, 8-color, 7-bit) cimbar, the respective numbers are `10850` and `8750`
* error correction level is `N/155`. So `ecc=30` corresponds to a `30:125` ratio of error correction bytes to "real" bytes.
* error correction is (for now) done via Reed Solomon, which contibutes to the rather large ratio of error correction bytes. See [ABOUT](ABOUT.md) for more technical discussion.

Expand Down
Loading
Loading