Simplified access to base lsd computation

[sheridana]

@pattonw just putting here for clarity. It would be nice to have a simplified repo / package that just does:

segmentation -> lsds

and nothing else.

couple things that could be added:

• make variance norm optional
• option for computing derivative lsds (should be faster)
• torch/jax implementation?

[vaxenburg]

Maybe also segmentation -> affinities?

[sheridana]

@vaxenburg see lsd-lite and the tests for examples. will add to pypi and add to lsd docs when i get a chance

[vaxenburg]

Thanks but I think the repo is private?

[sheridana]

sorry should be public now

[vaxenburg]

I think it looks good! I'll try it out soon.

I'm all for isolating the logic of as many things as possible into simple isolated light-weight callables, and with minimal dependencies. Maybe some of the gunpowder nodes, like the one that does label balancing, can be potential candidates too?

[sheridana]

could make sense to have a similar lightweight repo for common training functions, maybe @pattonw has thoughts

[pattonw]

Thats basically what the goal of the dacapo toolbox is now. I think Arlo adapted the lsd and aff implementation from there as well.
The main difference being that the toolbox is focused on tools for training with pytorch, so those methods are written to take torch tensors as inputs/outputs, wheras the lsd lite repo is using numpy right?
Weight balancing similar to gunpowder is in there: https://github.com/pattonw/dacapo-toolbox/blob/61b1c500a9ce68510cb56e544ea317b897a74794/src/dacapo_toolbox/transforms/weight_balancing.py#L6

[sheridana]

yeah exactly, I was thinking just something lightweight with numpy (and possibly scipy) as the only dependency. we discussed for the lsd-lite repo that it made more sense to just handle numpy arrays and let the user cast to whatever is needed in their training setup. but maybe dacapo toolbox is already fine @vaxenburg

[vaxenburg]

I, too, was thinking of lightweight numpy-only functionality actually, separate from any ML

[sheridana]

Okay so @pattonw a couple options - could just extend lsd-lite functionality, or just create a new one like dacapo-toolbox-lite and have all the relevant funcs as np implementations there. I don't think it makes sense to refactor the current dacapo-toolbox to be np only since it contains a lot of torch logic already.

[pattonw]

Yeah I think for now there will just have to be multiple implementations. Unfortunately torch tensors, numpy arrays, tensorflow tensors, jax arrays and others that I'm sure some people will want are not fully interchangeable.
For now I think an lsd-lite repo and a dacapo-toolbox repo are fine to cover numpy and torch use cases

[sheridana]

sounds good. @vaxenburg was the label balancing the only thing or were there also other gunpowder nodes you think would be useful?

[vaxenburg]

Yeah good question.. Maybe some of the augmentations too? Like the elastic (deform) augment maybe. The main use case I see for them is easy experimentation outside of the gunpowder pipeline. For example, If I wanted to experiment with the augmentation logic in isolation, to figure out suitable augmentation parameters for instance, such a stand-alone light-weight "node" would be quite helpful, I think?

[sheridana]

I'll defer to @pattonw on that, I think nodes like the deform augmentation gunpowder actually do quite a bit of the heavy lifting of ensuring roi compatibility that might be out of scope for extracting into a simple callable. And then other more simple nodes like noise augmentations simply wrap an external library function (for example, skimage.util.random_noise) and would therefore be overkill to put into a separate library. Maybe there is a good way to unify some of this but we should probably think about the structuring

[pattonw]

Yeah I think any of the spatial transforms (SimpleAugment/DeformAugment) are going to be hard to extract from gunpowder into standalone callables. Those, along with a lot of the Random Sampling require the concept of Rois, upstream Specs, BatchRequests and Requests.
We could potentially think about cutting down gunpowder to be these basic operations and then move gunpowder.torch and most of the nodes into separate libraries to make gunpowder itself light weight. But I think there are quite a few people using gunpowder so I'm a bit more hesitant about incorporating breaking changes there.