Color Quantization (breaking images into regions/clusters)

[Krasner]

This is a pretty involved topic. Kmeans is pretty slow on it's own - so brainstorming ways to speed things up.
Found this paper using superpixels as part of the quantization process:
https://www.mdpi.com/1424-8220/22/16/6043

I can try implementing SLIC for superpixels https://www.epfl.ch/labs/ivrl/research/slic-superpixels/

This can potentially replace both kmeans and mergeSmallRegionsInPlace

[Ryan-Millard]

Introduction

This is a very tough one, and I think it will take a lot of work to find the perfect solution since there are so many options out there.

While doing the initial research for this project, I chose the K-Means implementation over other options like Mean Shift and superpixel implementations because it would grant us speed - that choice still has some authority but it has weakened over time since the trade-off is large and it probably would be more beneficial to opt for a more accurate algorithm.

My Thoughts

Currently, my choice leans more towards a superpixel implementation (like SLIC) because Mean Shift is too close to our current implementation (K-Means) than SLIC would be. That, however, does not mean that I am making the correct choice - thus, we should research the possible options and propose a final solution.

@Krasner, if you'd like to implement it now, I recommend a quick implementation in something like Python to get the output faster. This will avoid possibly wasting your time on experiments and enable you to rapidly test different ideas and post their outputs here.

[Ryan-Millard]

mergeSmallRegionsInPlace is a very bad attempt at covering up a real issue, and I think SLIC could solve that.

[Krasner]

slic.ipynb

[Ryan-Millard]

What about SLIC++ - isn't it a bit better because it is more edge-aware?

[Ryan-Millard]

Your implementation looks perfect - the only issue I see is that it seems to always want to use the same kind of superpixel shape, which can cause inorganic-looking structures inside the image. K-Means will help with fixing that, but not completely.

I think the entire image processing pipeline may need to be refactored to emphasize edges since our current implementations are struggling with them.

[Krasner]

Yeah let's see how SLIC++ does....

[Ryan-Millard]

Would you be able to implement it? I'm currently researching the best contour tracing algorithm so the image can be converted to an SVG (it's important because that is the last step before Img2Num can become a proper package).

[Krasner]

yes

[Ryan-Millard]

@Krasner, just to let you know: if you find strange things happening to the image (like holes) in SVG format, that is a problem with imagetracerjs and I intend on having a custom raster-to-vector implementation as a result.

All that is left to be done for that are contour extraction and mapping those contours to SVG elements.

[Ryan-Millard]

I created #211 for this issue.

See this paper to understand why I think SLIC++ may be better in this scenario.

[Ryan-Millard]

That looks great - way closer to the original image.

[Ryan-Millard]

You seem to pick up these algorithms and implement them really fast! Have you worked with them a lot before?

[Krasner]

Some what but getting a lot of help from chatgpt 😆

[Ryan-Millard]

Damn, okay. When I use ChatGPT, it feels like it intentionally makes mistakes to waste my time.😂

[Krasner]

It definitely requires some intense review. But it provides a decent starting point...

[Ryan-Millard]

Does your IDE/text editor load .editorconfig files? I noticed some strange formatting, like spaces on empty lines.

[Krasner]

No but can look into it. I'm using VS code

[Ryan-Millard]

VS Code should support it. Knowing them, you probably need to add it to your stockpile of extensions.😂