Description
Tested version: 25.0.0 ea.04 (CE and Oracle JVMs)
(apologies for the length in advance)
We are running an experiment to investigate whether we can replace or augment our embedded Python technology in products with Graal. Our product has significant amounts of legacy Jython code -- a good amount is not written by us and we must maintain some customer compatibility with minimal changes -- and obviously both our customers and our internal teams therefore expect some Jython semantics to be in place. We've accepted there may be significant differences but we are confident that we can work around the vast majority of them with decorators or modules. However the code relies on some behavior that is not present. These all might be the same issue underneath.
Java exception handling
On previous versions (~24.1) this was simply not working - the objects you got back from:
try:
raise JavaException
except JavaException as e:
print(type(e).mro())
.. or thrown by internal Java classes were not valid objects and had strange behavior in that it was a ForeignObject etc. that had none of the expected BaseException semantics, nor as it would have been in Jython, been a carbon copy of the Java Exception (up to and including any custom methods).
We are very pleased with the changes moiving from 24.1.1 to 25.0.0 ea.04 in that now we do not need to supply --python.EmulateJython=true to get anything to work, and that we get back as 'e' a very Pythonic BaseException-based object.
Unfortunately we have exceptions thrown by our own internal classes which have custom methods (e.g. OurException.getErrorCode() returns a number as encoded in the message string, since we have a library that acts a little bit like strerror() and comes up with a much more productive string. Don't judge.. it is an old bit of code). There seems to be no way to fetch the original Java exception object or access it as we would in Jython here.
Note that the documentation here doesn't seem correct:
.. in that
a) --python.EmulateJython=true is not necessary at all (we like this behavior, although either way b occurs)
b) e.getMessage() does not work in the example as e is not the original Java exception anymore.
The code in the example in the docs is exactly our litmus test for it.
Given that we want to call random unhealthy methods on Exception classes which is not very standard, Pythonic or Java-ic of us, this is something we would be happy to accept needs a little bit of porting. We did expect that e.super or something of that ilk would give us the original class but it does not.
Summary: Semantics are different from Jython in that the Exception object caught does not have the methods or members of the original Java Exception object. There is no way to walk the MRO or find the original object to access those methods.
Java subclassing
The documentation here (commit-ish is the master branch as of today):
Has a very confusing example. We have found that while this example works fine, there are two major issues (and one minor) with it.
a) (Minor) This is absolutely not compatible with Jython, although this is stated in the documentation. We also noted that --python.EmulateJython does not change the behavior at all implying this is not a Jython-emulation-specific feature in Graal. In this code:
from com.example.h2g2 import DeepThought
class MyDeepThought(DeepThought):
def __init__(self):
self.answer = 42
def reveal_answer(self):
print(f"the answer is... {self.answer}")
dt = MyDeepThought()
dt.reveal_answer()
Does not work. If you attempt to call a method on the Python side, you can't because it is actually
dt.this.reveal_answer()
This alteration does work, and in fact we think we could make a decorator to smooth this out a bit. It goes downhill from here.
b) Augmenting the class further we have to use our imagination and say that the original Java class has a simple member "question" and a method "guide_entry":
class MyDeepThought(DeepThought):
def __init__(self):
self.answer = 42
# self.__super__.guide_entry("HARMLESS") # NoneType is not callable
# self.__super__.__init__() # NoneType is not callable
super().__init__() # this works, though
self.guide_entry("HARMLESS") # this does too
def guide_entry(self, arg): # this is overriding a method on Java class DeepThought
print(f"{arg}")
self.__super__.guide_entry(arg) # works ok?
def reveal_answer(self):
print(f"the answer is... {self.answer}")
def reveal_question(self):
# qu = self.this.question # wrong, AttributeError: 'PythonJavaExtenderClass' object has no attribute 'this'
# qu = self.question # wrong, AttributeError: 'PythonJavaExtenderClass' object has no attribute 'answer'
print(f"the question is... ") # should print: "how many roads must a man walk down?"
dt = MyDeepThought()
dt.guide_entry("MOSTLY HARMLESS") # prints "MOSTLY HARMLESS" and then calls the original Java method (yay!)
dt.this.reveal_answer()
dt.this.reveal_question() # will cause exception as above depending on impl.
print(f"{dt.question}") # will print the member value as intended
Essentially, we cannot access any Java members from inside the class, but methods are fine.
Those members are published outside the class definition on the object as constructed.
This feels as if the class is created, and then GraalPy is wrapping the class and adding a few members (graalpython.build_java_class() or .extend()?) but the user (whoever owns the 'dt' instance above) and the class (receiving 'self') are getting two different objects or different wrappings? Whatever the cause, it is inconsistent.
c) Additionally we can only call super methods which match the name of the subclassed function? Understandably having one function call a superclass method for a different method is a little strange but that is a programmer problem and we're not sure it should be enforced by the "language" in that you can do it in CPython.
If we try and run something from the __init__ method we also find that self.__super__ is None. Luckily, super() works. This is a deviation from the documentation (which is very good, we like that it works like Python).
I have not had time to express all the possibilities here in code to make sure.
Jython functionality might not be able to be fine-grained enabled/disabled
We are very pleased with the direction this is all taking except for the issues above. However we have to express a little fear of progress here, if features like "not having to EmulateJython=true for either of the above to be possible" are not intentional and this is all going to be hidden behind that feature, we really would like to have a more fine-grained enablement. As far as we're aware the Jython emulation has traditionally intended to bring in 3 features:
- Features like jarray (although this has always seemingly worked without emulation ✅ )
- Subclassing Java classes (although this only works properly in 25.0.0, and works without the emulation now ✅ )
- Providing Exceptions (although this only works in properly 25.0.0, and works without the emulation now ✅ )
- Exposing a renamed property when it detects a getter or setter in a class, e.g. if getMyInternalVariable() and setMyInternalVariable() exist then myclass.myInternalVariable will also exist (if get*() exists alone then it is read-only). ❌
We do not like the last feature as it confuses the namespace and in general makes our docs look incomplete, and we would like to disable such functionality while keeping others if at all possible. We considered patching Jython to disable this feature about a decade ago but it didn't seem worth the trouble to maintain it. In that sense, we would not like to see it back. We're looking for a very "clean Python-like" environment with implicit Java compatibility and no API changes.
I suppose this is a plea to see if we can find out if there is a goal for this or a JSR or JEP which covers what the API should look like, and what the behavior is. We are happy to work with the team here in figuring this out, so that we can plan accordingly and come back at a future version, or test more in the meantime.