Fix issue #20 by specifying explicit target path for depencies install

[ChrisMzz]

See my comment on #20.

It seems that on Windows, for some reason Python looks at site-packages from distributions elsewhere in the computer, notably in AppData/Roaming/Python.
I think this is fixed by specifying explicitly the target path for the dependencies to be installed in:

python_path = get_python_path()
target = Path(python_path).parent.parent/"lib"/"site-packages" # in Blender's Python distribution
packages = {'grpcio', 'bidict', 'grpcio-tools', 'pandas'}
subprocess.check_output([python_path, '-m', 'pip', 'install', "--upgrade", '--target', str(target), *packages])

Since this path was implicit in situations where this problem doesn't occur, this change should be inconsequential to those situations.

From what I've tested on my end this works in both the situation with the issue and the situation without the issue.

[ChrisMzz]

Just saw you assigned me to this @stefanhahmann and noticed that your get_python_path checks for python in bpy.app.binary_path_python only if the version is 2.92 or lower.

I tested this PR by making sure the installation would work when running blender.exe --background --python install_addon.py (+ the zip file), but I'm now wondering if the code will run using Blender's Python executable or not when installing directly from the Mastodon Launcher.

Do you have a way of testing this? I'm not much of a Java person so I'm not sure how to compile into a jar, although I'm fairly certain this change I made will not fix the problem if the path to the python executable doesn't point to Blender's local files.

[stefanhahmann]

@ChrisMzz Thank you very much for your contribution and sorry for not having it seen earlier.

I have to admit that I took over the task of maintaining this repo, but I did not actually develop this. This was mainly done by @maarzt

My way of testing would be:

• Go to terminal

• Navigate to root folder

• Check out branch

• Run mvn package from root folder (needs Java and maven to be installed on the machine)

• Find new *.jar file in folder target

• Copy this *.jar file to a Fiji installation that has Mastodon and Mastodon Blender View

• Delete the existing Blender View jar

• Test, if the connection to Blender can be made via the dialog inside Mastodon

We unfortunately do not have an automated test for this. May I ask you to run through this procudure on your device. If this is successful, I would try to run it through on a Windows machine that I have access to (and perhaps also a Mac). If all of this is successful, then the PR is ready to be merged.

For the current state of the code, I have already compile the jar: mastodon-blender-view-0.3.5-SNAPSHOT.jar.zip

[ChrisMzz]

Thanks for the instructions (good to know in general for Java projects too!), I was able to compile my jar into a 0.3.5-snapshot as well, with the modifications from the PR. To be honest I think I forgot about this:

String output = runCommandGetOutput( blenderPath.toString(), //
		"--background", //
		"--python", pythonScript.toAbsolutePath().toString(), //
		"--", addonZip.toAbsolutePath().toString() ); //

which quite explicitly does the exact same thing as my manual test as I was trying to resolve the issue. I guess it's been a little while.
It still didn't hurt to check with the new jar file, so I once again made sure installation was failing with both mastodon-blender-view-0.3.4 and the snapshot you sent me, then replaced the plugin with this new compiled jar and ran the installation from the Mastodon Launcher. Sure enough this worked!

I made sure to check in the Blender installations I selected through the setup, that the site-packages folder from the local Python distribution did not have the dependencies before installing and that they were properly installed by trying to install the plugin, since that was the main source of the issue.

For reference:

• Tested on both Blender 4.2.16 and 5.0.1, using ImageJ 2.16.0
• I'm testing on Windows 11, and compiled the jar with JDK-25
• Conditions to reproduce (on Windows): have an active Python distribution in %APPDATA%/Python, and a standalone Blender distribution that doesn't already have the required packages in its local python installation, which could be due to another plugin having been installed for Blender (the easiest way to make sure this is true is just to install a new standalone Blender distribution).

[stefanhahmann]

To be honest I think I forgot about this:

String output = runCommandGetOutput( blenderPath.toString(), //
		"--background", //
		"--python", pythonScript.toAbsolutePath().toString(), //
		"--", addonZip.toAbsolutePath().toString() ); //

which quite explicitly does the exact same thing as my manual test as I was trying to resolve the issue.

Thanks for testing. May I ask you to modify the PR such that these further changes are included as well?

[ChrisMzz]

Oh sorry if it wasn't clear, this isn't a change, it's in L61 in BlenderSetupUtils.java file that is already in the source code. The reason I brought it up is to specify that the installer uses Blender's python installation already, so there's no additional checks needed.

If you approve the PR and compile the jar with the code as it is currently it should produce a valid jar that fixes the issue :)

[Copilot]

Pull request overview

This PR fixes issue #20 where Python package dependencies were being looked up from incorrect locations on Windows (such as AppData/Roaming/Python instead of Blender's Python distribution). The fix explicitly specifies the target installation path for pip to ensure dependencies are installed in Blender's bundled Python environment.

Changes:

• Added explicit --target flag to pip install command pointing to Blender's Python site-packages directory
• Added --upgrade flag to ensure packages are updated if already present
• Imported pathlib.Path for cross-platform path construction

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[maarzt]

Thank you @ChrisMzz for investigating this issue so thoroughly and providing a PR. And thank you @stefanhahmann for reviewing this issue further. Here is a short summary of what I got:

• On Windows under some circumstances the install script installs packages in the wrong directory.
• And adding the --target flag to the pip install command with a correct "site-packages" folder fixes the problem.

Unfortunately the suggested change breaks the install script on linux (Ubuntu). So we cannot merge this PR as it is. The correct paths on my machines are:

/home/arzt/Applications/blender-5.0.1-linux-x64/5.0/python/lib/python3.11/site-packages
/home/arzt/Applications/blender-5.0.1-linux-x64/5.0/python/bin/python3.11

So the script constructs the wrong path.

With all the provided info. I very much wonder if this is related the "Portable Blender" requirement. Mastodon is - for now - only expected to work with a "Protable Blender" installation. I hope you already double checked this. But just to be sure. I have to ask. Blender comes in two variants for Windows. Do you use the portable version of Blender: Blender Portable (.zip) and not the standard Blender Installer.
See https://www.blender.org/download/:

How this could be fixed:

• Verify if using Portable Blender fixes the problem or not.
• Let's find out what are the proper install directories in Windows, Mac, Linux. Where blender.exe, python.exe and the site-packages folders. This would be interesting for both the portable and non-portable Blender. @ChrisMzz, @stefanhahmann Could you let me no what the paths are on your machines.
• Find a way to construct the correct install path. Maybe use site.getsitepackages().
• Fix the install_addon.py script. Maybe move the installation entirely inside the Blender plugin.

[ChrisMzz]

I did indeed use the portable version of Blender.
I have just checked that import site; print(site.getsitepackages()) does indeed return the correct path to Blender's site packages on Windows at least, which for reference is

<path_to_Blender>\blender-5.0.1-windows-x64\5.0\python\lib\site-packages

but it returns other things as well, so I need to test this a bit.

My bad for including --upgrade, it didn't occur to me that this could break other plugins. I will commit the relevant changes in a second.

[maarzt]

@ChrisMzz wrote:
I did indeed use the portable version of Blender.

Thank you for double checking! To be honest. I'm very surprised that Blender runs in a python installation where pip install fails by default. Thank you for all your detective work to get to the root of this.

I have just checked that import site; print(site.getsitepackages()) does indeed return the correct path to Blender's site packages on Windows... but it returns other things as well, so I need to test this a bit.

... interesting. Maybe pip uses one of the folders that don't work. Your updated install_addon.py will likely work. However it still feels a bit improvised. I found other online discussions that suggest to install dependency inside the addon folder. And this gets me exciting. Maybe we get mastodon Blender to work with any Blender installation. No longer requiring "portable" installation.

My bad for including --upgrade, it didn't occur to me that this could break other plugins. I will commit the relevant changes in a second.

No worries. To be honest, I'm not sure whether having this --ugrade flags helps or breaks things. But when it comes to python environments, my approaches is closer to "never change a running system".

[maarzt]

Small fix to have the installation succeed even if a "site-packages" folder is not found.

[maarzt]

While verifying that we use pip install correctly in the connection with Blender, and especially trying to get an answer to the question: What path to pass to the --target flag. I came across this repo blender-python-example. It suggests a different solution that will likely fix #20. The only thing required is to pass an environment variable PYTHONNOUSERSITE=1 to pip install. That way pip should install the dependencies into the correct folder.

I think I was able to reproduce the problem on my computer. And the fix PYTHONNOUSERSITE=1 worked as expected.

@ChrisMzz Sorry to ask you for another test run. Could you verify that this new solution works on you machine as well. That would be awesome.

Remark: It's worth noting, that Mastodon Blender View, is not a well behaved Blender plugin. As it breaks the Blender Add-on Guidelines. Blender Addon should not call pip at all.

[maarzt]

Suggested change

	from pathlib import Path
	import site

not required anymore

[maarzt]

Suggested change

	def get_target_flags():
	# On Windows pip will sometimes install dependencies to the wrong folder when not setting the target flag.
	# See: https://github.com/mastodon-sc/mastodon-blender-view/pull/36
	for path in site.getsitepackages():
	if "site-packages" in path:
	return ["--target", path]
	return []

not required

[maarzt]

Suggested change

	target_flags = get_target_flags()
	packages = ['grpcio', 'bidict', 'grpcio-tools', 'pandas']
	subprocess.check_output([python_path, '-m', 'pip', 'install', *target_flags, *packages])
	env = dict(os.environ)
	env["PYTHONNOUSERSITE"] = "1"
	packages = {'grpcio', 'bidict', 'grpcio-tools', 'pandas'}
	subprocess.check_call([python_path, '-m', 'pip', 'install', *packages], env=env)

We don't need to provide a --target flag. Passing a PYTHONNOUSERSITE is apparently the preferred solution, as demonstrated her https://github.com/robertguetzkow/blender-python-examples/blob/92214a9d74eedad8572e890a374d5204623c796a/add_ons/install_dependencies/install_dependencies.py#L114-L117

[maarzt]

@stefanhahmann Could you try this mastodon-blender-view-0.3.5-SNAPSHOT.jar works for you as well?
(It has a modified install_addon.py script for the installation of the Blender Addon. It passes a PYTHONNOUSERSITE=1 environment variable to pip install.)

Steps for testing:

• Install a fresh copy of a "portable" Blender.
• Optional: Maybe run something like <path to blender>/5.0/python/bin/python3.11 -m pip install --user bidict
• Delete mastodon-blender-view.jar from the jars folder in Fiji.app
• Copy this modified mastodon-blender-view-0.3.5-SNAPSHOT.jar to the jars folder in Fiji
• Restart your Fiji
• Open Mastodon project.
• Setup Mastodon Blender Plugin
• Try Interactive Blender View and Advanced Blender View

[ChrisMzz]

No worries, I will try to test the new jar this afternoon. Even if it currently breaks the addon guidelines, it's preferable if it installs as intended for all users, independently of their OS. If the jar works for both me and Stefan should I then commit the changes you suggested?

[stefanhahmann]

@stefanhahmann Could you try this mastodon-blender-view-0.3.5-SNAPSHOT.jar works for you as well? (It has a modified install_addon.py script for the installation of the Blender Addon. It passes a PYTHONNOUSERSITE=1 environment variable to pip install.)

I did a test with mastodon-blender-view-0.3.5-SNAPSHOT.jar on MacOS. The test failed:

However, this may not be related to this PR and may have failed already with mastodon-blender-view-0.3.4, which I did not test.

Will do a test on Windows as well.

[stefanhahmann]

@stefanhahmann Could you try this mastodon-blender-view-0.3.5-SNAPSHOT.jar works for you as well? (It has a modified install_addon.py script for the installation of the Blender Addon. It passes a PYTHONNOUSERSITE=1 environment variable to pip install.)

I also tested on Windows.

My results:

When I first tried to install the addon, it ran more than two minutes. Then I cancelled the dialog. After I went to the dialog again, it said "Addon installed".

The test said "Addon successfully tested".

However, when I tried to open the interactive Blender View the following happened:

I used BlenderPortable 5 for the test.

[maarzt]

@stefanhahmann Oh no, that's not good. I need to sit down with you to get an idea what is going on. Unfortunately I'm busy with a lot of other projects.

[stefanhahmann]

However, this may not be related to this PR and may have failed already with mastodon-blender-view-0.3.4, which I did not test.

Just wanted to report that I tested with mastodon-blender-view-0.3.4 on MacOS. With this version everything works as expected. Thus, I conclude that there is problem on MacOS with mastodon-blender-view-0.3.5-SNAPSHOT.