Skip to content

【bug】PythonPath lib invalid when initialize PythonInterpreter #94

@raoraoxiong

Description

@raoraoxiong

bug briefly description:

The PythonInterpreter will initialize with exception "ModuleNotFoundError: No module named 'find_libpython' " when the PYTHON_HOME is a pure python interpreter (not include find_libpython) and the PYTHON_PATH include 'find_libpython' lib. It look likes that the PYTHON_PATH has no effects when initialize PythonInterpreter

This is a Flink TaskManager Error log when enable pemja mode

2026-03-06 10:50:26,740 WARN  [org.apache.flink.runtime.taskmanager.Task.transitionState(Task.java:1138)] [] - Source: source[1] -> PythonCalc[2] -> Sink: Unnamed (1/1)#0 (42b42f3ddcc7d2bd8f0fe59d3bf83f7c_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from INITIALIZING to FAILED with failure cause:
java.lang.RuntimeException: Failed to find libpython
	at pemja.utils.CommonUtils.getPythonLibrary(CommonUtils.java:161) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at pemja.utils.CommonUtils.loadPython(CommonUtils.java:44) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at pemja.core.PythonInterpreter$MainInterpreter.initialize(PythonInterpreter.java:365) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at pemja.core.PythonInterpreter.initialize(PythonInterpreter.java:144) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at pemja.core.PythonInterpreter.<init>(PythonInterpreter.java:45) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator.open(AbstractEmbeddedPythonFunctionOperator.java:72) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.table.runtime.operators.python.AbstractEmbeddedStatelessFunctionOperator.open(AbstractEmbeddedStatelessFunctionOperator.java:84) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.table.runtime.operators.python.scalar.EmbeddedPythonScalarFunctionOperator.open(EmbeddedPythonScalarFunctionOperator.java:89) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:107) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreStateAndGates(StreamTask.java:858) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$restoreInternal$5(StreamTask.java:812) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.call(StreamTaskActionExecutor.java:100) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:812) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:771) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:979) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:948) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:772) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584) ~[flink-dist-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.io.IOException: Failed to execute the command: /tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-archives/cpython39-venv.tar.gz/venv/bin/python -c from find_libpython import find_libpython;print(find_libpython())
output: Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'find_libpython'

	at pemja.utils.CommonUtils.execute(CommonUtils.java:180) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	at pemja.utils.CommonUtils.getPythonLibrary(CommonUtils.java:157) ~[flink-python-release-TDFLINK-131811631-v1.jar:release-TDFLINK-131811631-v1]
	... 18 more

Additional, I print the python interpreter config before initialize Python interpreter, the log show like this:

2026-03-06 10:50:25,330 INFO  [org.apache.flink.python.env.AbstractPythonEnvironmentManager.constructEnvironmentVariables(AbstractPythonEnvironmentManager.java:182)] [] - PYTHONPATH of python worker: /tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-files/101856c7-35c0-4ca4-a89e-6784fab28218_toolz_test.py:/tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-files/90576691-06db-4be9-9d84-3fd32654a041_tdflink-lib.zip/tdflink-lib:/tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-files/ad745872-8f94-4087-8162-963d732d8929_user-python-dependency-32227-1.zip/user-python-dependency-32227-1
2026-03-06 10:50:25,330 INFO  [org.apache.flink.python.env.AbstractPythonEnvironmentManager.constructArchivesDirectory(AbstractPythonEnvironmentManager.java:377)] [] - Python working dir of python worker: /tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-archives
2026-03-06 10:50:26,095 INFO  [org.apache.flink.python.env.AbstractPythonEnvironmentManager.constructEnvironmentVariables(AbstractPythonEnvironmentManager.java:201)] [] - Python interpreter path: cpython39-venv.tar.gz/venv/bin/python
2026-03-06 10:50:26,097 INFO  [org.apache.flink.python.env.embedded.EmbeddedPythonEnvironmentManager.createEnvironment(EmbeddedPythonEnvironmentManager.java:111)] [] - Print pemja config, pythonHome: /tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-archives/cpython39-venv.tar.gz/venv/, pythonExec: /tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-archives/cpython39-venv.tar.gz/venv/bin/python, pythonPath: [/tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-files/101856c7-35c0-4ca4-a89e-6784fab28218_toolz_test.py, /tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-files/90576691-06db-4be9-9d84-3fd32654a041_tdflink-lib.zip/tdflink-lib, /tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-files/ad745872-8f94-4087-8162-963d732d8929_user-python-dependency-32227-1.zip/user-python-dependency-32227-1], execType: MULTI_THREAD

The 'find_libpython' module exists in the python path:
/tmp/python-dist-1103fa2b-e5c8-4c33-86ba-c19993909bbc/python-files/90576691-06db-4be9-9d84-3fd32654a041_tdflink-lib.zip/tdflink-lib

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions