Skip to content

Appose branch: memory issue(s) #8

@NicoKiaru

Description

@NicoKiaru

First of all, the allocated RAM needs to be sufficient in order to load the full brainglobe atlases (it's not like a virtual stack). If you don't, you get the following error:

The default environment has been installed.
java.lang.RuntimeException: Failed to load BrainGlobe atlas: kim_dev_mouse_mri_t2_10um
	at ch.epfl.biop.atlas.scijava.AtlasChooserCommand.lambda$registerBrainGlobeAtlases$0(AtlasChooserCommand.java:133)
	at ch.epfl.biop.atlas.scijava.AtlasChooserCommand.run(AtlasChooserCommand.java:92)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:125)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:64)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:247)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: org.apposed.appose.TaskException: Task failed: Traceback (most recent call last):
  File "C:\Users\Nicolas\.local\share\appose\brainglobe-abba-2.3.0\.pixi\envs\default\Lib\site-packages\appose\python_worker.py", line 121, in _run
    exec(compile(block, "<string>", mode="exec"), _globals, binding)
  File "<string>", line 39, in <module>
  File "<string>", line 33, in to_shared
  File "C:\Users\Nicolas\.local\share\appose\brainglobe-abba-2.3.0\.pixi\envs\default\Lib\site-packages\appose\shm.py", line 135, in __init__
    SharedMemory(
  File "C:\Users\Nicolas\.local\share\appose\brainglobe-abba-2.3.0\.pixi\envs\default\Lib\site-packages\appose\shm.py", line 41, in __init__
    super().__init__(name=name, create=create, size=rsize)
  File "C:\Users\Nicolas\.local\share\appose\brainglobe-abba-2.3.0\.pixi\envs\default\Lib\multiprocessing\shared_memory.py", line 131, in __init__
    h_map = _winapi.CreateFileMapping(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 1450] Insufficient system resources exist to complete the requested service: 'wnsm_6348bec0'

	at org.apposed.appose.Service$Task.waitFor(Service.java:729)
	at ch.epfl.biop.atlas.brainglobe.BrainGlobeAppose.fetchAtlas(BrainGlobeAppose.java:243)
	at ch.epfl.biop.atlas.brainglobe.BrainGlobeAtlas.initialize(BrainGlobeAtlas.java:89)
	at ch.epfl.biop.atlas.scijava.AtlasChooserCommand.lambda$registerBrainGlobeAtlases$0(AtlasChooserCommand.java:130)
	... 9 more
No atlas was returned - the open atlas command did not work as expected.

If you put enough RAM, you get to the second issue which is that the shared memory is backed by a ByteBuffer and its size is capped to 2Gb. So you get this error:

java.lang.RuntimeException: Failed to load BrainGlobe atlas: kim_dev_mouse_mri_t2_10um
	at ch.epfl.biop.atlas.scijava.AtlasChooserCommand.lambda$registerBrainGlobeAtlases$0(AtlasChooserCommand.java:133)
	at ch.epfl.biop.atlas.scijava.AtlasChooserCommand.run(AtlasChooserCommand.java:92)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:125)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:64)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:247)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalArgumentException: Negative capacity: -1887287296
	at java.nio.Buffer.<init>(Buffer.java:199)
	at java.nio.ByteBuffer.<init>(ByteBuffer.java:281)
	at java.nio.ByteBuffer.<init>(ByteBuffer.java:289)
	at java.nio.MappedByteBuffer.<init>(MappedByteBuffer.java:89)
	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:162)
	at com.sun.jna.Native.getDirectByteBuffer(Native Method)
	at com.sun.jna.Pointer.getByteBuffer(Pointer.java:654)
	at org.apposed.appose.shm.ShmBase.buf(ShmBase.java:79)
	at org.apposed.appose.NDArray.buffer(NDArray.java:109)
	at net.imglib2.appose.NDArrays.asArrayImg(NDArrays.java:118)
	at net.imglib2.appose.NDArrays.asArrayImg(NDArrays.java:83)
	at net.imglib2.appose.ShmImg.<init>(ShmImg.java:79)
	at ch.epfl.biop.atlas.brainglobe.BrainGlobeAtlasMap.ndArrayToSourceAndConverter(BrainGlobeAtlasMap.java:137)
	at ch.epfl.biop.atlas.brainglobe.BrainGlobeAtlasMap.initializeFromApposeData(BrainGlobeAtlasMap.java:89)
	at ch.epfl.biop.atlas.brainglobe.BrainGlobeAtlas.initialize(BrainGlobeAtlas.java:96)
	at ch.epfl.biop.atlas.scijava.AtlasChooserCommand.lambda$registerBrainGlobeAtlases$0(AtlasChooserCommand.java:130)
	... 9 more
No atlas was returned - the open atlas command did not work as expected.

I don't know how the memory was handled with imglyb and pyimagej, but I did not have these issues. I'm not sure which direction I should follow to get this brainglobe atlas wrapping to work with Appose.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions