ZuriHac 2020 - resize

[kahlil29]

Hey,
I came across this listed on the ZuriHac website and I wanted to know if there are any specific issues that you have in mind for the hackathon?
I don't have much experience with Image Processing (libraries) but recently I had to write programs (at work) for image resizing in Rust, Go, Python and Haskell and found Haskell seems lacking in this area (in terms of a feature complete Image Processing Library).

I would like to try and help in some way to do something about that situation.

[lehins]

@kahlil29 The plan is to rewrite the whole library using massiv library #30, I started already a while back, which is in https://github.com/lehins/hip/tree/massiv, but had to stop for a while because I didn't have all of the prerequisites I needed.

Resizing is already implemented, but it is far from perfect, but besides the need for it to be rewritten with massiv it also needs fixing.

hip/src/Graphics/Image/Processing/Geometric.hs

Lines 369 to 386 in 2bbdda7

	resize :: (ColorSpace cs e, Interpolation method) =>
	method -- ^ Interpolation method to be used during scaling.
	-> Border (Pixel cs e) -- ^ Border handling strategy
	-> Ix2 -- ^ Dimensions of a result image.
	-> Image cs e -- ^ Source image.
	-> Image cs e -- ^ Result image.
	resize method border sz'@(m' :. n') (Image arr) = Image (A.compute (A.traverse sz' getNewPx arr))
	where
	sz@(m :. n) = A.size arr
	!fM = fromIntegral m' / fromIntegral m
	!fN = fromIntegral n' / fromIntegral n
	getNewPx getPx (i :. j) =
	interpolate
	method
	(A.handleBorderIndex border sz getPx)
	((fromIntegral i + 0.5) / fM - 0.5, (fromIntegral j + 0.5) / fN - 0.5)
	{-# INLINE getNewPx #-}
	{-# INLINE resize #-}

Your expertise on this, having written it in other languages, will be extremely valuable. Let's talk on discourse about it. If you can't fin me there I will always respond on https://gitter.im/haskell-massiv/Lobby

[lehins]

@kahlil29 Are you still interested in working on this? Ping me on Discord#hip if you are. Even if you are not ping me anyways I'll be happy to pick your brain about your past experience implementing resize.

[kahlil29]

Hey @lehins
I think I may have mislead you with my previous message.
I didn't actually write resize implementations, but merely programs to resize images (using the respective image libraries in each language).

I'm still interested in contributing in some way to hip or massiv and I've joined the #hip discord channel in ZuriHac. Looking forward to interacting with all of you there 😄

[lehins]

No worries, that knowledge will be useful as well. For example here are some questions I have:

• I know there are a couple implementations of resize functionality in Haskell, including one in HIP, what do you think was lacking in them? I know the current limitations in HIP, which I want to fix, but I wanted to hear your opinion on this.
• Are the implementations you did open source? I'd be curious to look at the uses cases in other languages.
• Could you post here a list of links to the documentation for resize functions you used in those languages. It will be great to compare to how other languages

[kahlil29]

• So for the project at work, performance was the biggest goal since it's expected to have high load at certain peak times.
  I just went back and verified that we used JuicyPixels, not hip for resizing in the Haskell program. Cannot remember why, though. Is it possible that hip got resizing or additional resizing features more recently? Because that program in Haskell was written maybe a year or more ago.

This may be a Haskell issue rather a library or implementation issue but the performance was not very fast, compared to Rust (image crate), although Rust is known to be faster, in general.
Finally ended up going with Pillow-SIMD which was almost 2x as fast as Rust (and Rust's image crate resizing was almost 2x as fast as our Haskell program).

Here's a performance comparison we did yesterday (as we have just finished writing the Rust and Python versions of these, but do not take this graph very seriously as we may have fine tuned the Python programs a bit more once we realized it was way faster and we would end up going with it)

• Apart from performance, I would say that the next thing that comes to mind is the number of resizing (interpolation) methods available. Posting the links to some Go libraries (there seem to be quite a few of these) to indicate features and examples of good documentation (appealing to a user at first glance):
  Bild
  Imaging
  Resize

Also, we could have a quick table listing the image formats that are currently supported.

• Regarding the code, I'll try and get back to you with some links or I'll create some gists with the programs, they're really very simple programs that involve 3 steps:

  1. Download Image from AWS S3
  2. Resize Image
  3. Upload Image to AWS S3

• Documentation links:
  Pillow
  Image crate (Rust)
  OpenCV-Python

I hope this is somewhat helpful and apologies if there's too much focus on just resizing from me, I didn't do much more in my code, but I would be open to reading the docs/going through the repos of the libraries linked above if we want a better idea of things that hip is missing that could be added (if this is feasible).

[lehins]

@kahlil29 This is great and definitely very helpful. Focus on resize is exactly what this ticket is about! ;)

I benchmarked yesterday current implementation that is linked in this #39 (comment) and it is already 3 times faster than JuciyPixels just because massiv provides automatic parallelization.

I would be very greatful if you could provide some gists that do resizing in other languages, this would help me optimize the current implementation in HIP.

[kahlil29]

That's amazing to hear (the 3x faster bit) and happy to be of some help 😄
Just to clarify, you want gists of programs that import and call resize functions (in other langs) or you want to see the source code from the libraries of those implementations?

[lehins]

gists of programs that import and call resize functions (in other langs)

That's the bit I am most interested in, because it will let me benchmark against those languages.