Fixes for compatibility with Python 3.13 #1321

mmatera · 2025-01-28T19:45:06Z

This PR contains the fixed needed to make Mathics compatible with Python 3.13.

mmatera · 2025-01-28T19:46:25Z

mathics/builtin/files_io/importexport.py

-     | 0.838545	0.247025
-     | 0.838697	0.436220
-     | 0.309496	0.833591
+     | 0.629452    0.586355


In docstrings, use 4 spaces instead of a tab character.

mmatera · 2025-01-28T19:48:24Z

.github/workflows/ubuntu.yml

@@ -11,7 +11,7 @@ jobs:
    runs-on: ubuntu-latest
    strategy:
      matrix:
-        python-version: ['3.12', '3.11', '3.8', '3.9', '3.10']
+        python-version: ['3.13', '3.12', '3.11', '3.8', '3.9', '3.8']


Should keep 3.8? Maybe 3.8, 3.10, 3.12 and 3.13 would be enough?

I am for dropping 3.8.

OK, so ['3.13', '3.12', '3.11', '3.10', '3.9'] in all the workflows?

mmatera · 2025-01-28T19:49:19Z

mathics/core/builtin.py

@@ -468,6 +468,8 @@ def get_functions(self, prefix="eval", is_pymodule=False):
        for name in dir(self):
            if name.startswith(prefix):
                function = getattr(self, name)
+                if not hasattr(function, "__call__"):


Now, NoneType has a docstring, so we need to check that function is indeed a function.

mmatera · 2025-01-28T19:49:48Z

mathics/doc/doc_entries.py

@@ -93,7 +93,7 @@
    r"""(?mx)^  # re.MULTILINE (multi-line match)
                # and re.VERBOSE (readable regular expressions
        ((?:.|\n)*?)
-        ^\s+([>#SX])>[ ](.*)  # test-code indicator
+        ^\s*([>#SX])>[ ](.*)  # test-code indicator


In Python 3.13, doctrings are trimmed of "margin" spaces.

mmatera · 2025-01-28T19:50:47Z

mathics/doc/doc_entries.py

-
-    # Remove leading <dl>...</dl>
-    # doc = DL_RE.sub("", doc)
+    doc = filter_comments(doc)  # .strip("\s")


I do not remember the reason of the r"\s" strip, but it breaks some tests now.

mmatera · 2025-01-28T19:51:43Z

mathics/doc/doc_entries.py

@@ -394,11 +391,16 @@ def compare_out(self, outs: tuple = tuple()) -> bool:
            # Mismatched number of output lines, and we don't have "..."
            return False

+        # Python 3.13 replaces tabs by a single space in docstrings.


Python 3.13 converts tab characters in docstrings to simple spaces. This "hack" helps to keep the doctests compatible with this change.

rocky · 2025-01-28T21:36:31Z

Note: Supporting 3.13 may push us into using a newer pip that disallows the find_namespace_packages that is needed right now in the Mathics3 Modules.

Keep in mind, getting everything working on 3.13 where we can release might be a bigger undertaking than just getting the Mathics Kernel adjusted.

mmatera · 2025-01-28T21:41:40Z

OK, this is just to adjust the kernel compatibility, and does not breaks the compatibility with previous versions.

rocky · 2025-01-28T21:49:19Z

OK, this is just to adjust the kernel compatibility, and does not breaks the compatibility with previous versions.

And that means we cannot do a full release like we just did until everything is fixed.

rocky · 2025-01-28T21:50:16Z

.github/workflows/ubuntu.yml

@@ -11,7 +11,7 @@ jobs:
    runs-on: ubuntu-latest
    strategy:
      matrix:
-        python-version: ['3.13', '3.12', '3.11', '3.8', '3.9', '3.8']
+        python-version: ['3.13', '3.12', '3.10', '3.8']


Did you mean to change 3.8 to 3.9?

I was saying to keep the even versions, plus the last one (if odd).

3.8 is the version supported by the last version of Pyston.

I think we should drop Pyston then in its standalone form. There is still a pyston-lite plugin.

I don't see a reason to drop versions that end in an odd number or not test them.

Python does not say that minor numbers that end in odd numbers have any less status than those with even numbers. Dropping 3.8 makes sense because the Python language drifts and it is extra effort to support older Pythons.

It is a pitty because Pyston is much more faster than CPython and Pyston-lite, but OK. In any case, we still are compatible with 3.8

rocky · 2025-01-29T14:56:57Z

I don't have a problem with the idea or the code, but the timing of doing this right now is weird.

I've just spent the last 3 days if not longer trying to get out a release ensuring all the parts work together. (And I am not sure they completely do). Something like this now ensures things will be a little bit out of whack again and puts us back in the state of a little off-ness. I don't think some of the code will be able to handle 3.13 unless an older pip is used. And code in other repositories may need additional adjusting too.

And of course, there are other PRs in the backlog. These change the API which will also require changing some other bits of code outside of this repository.

To accommodate moving forward which we need to do and keep some stability for a little while right now, I suggest we have an "unstable" branch that we merge to, along with the master branch which is compatible across other projects.

Your thoughts?

rocky · 2025-01-29T15:02:32Z

Hmm. We could split this up into two parts, isolate that version testing change part and everything else. So that when we are ready to switch over everything there is a small bit of code in this repository that does that.

mmatera · 2025-01-29T15:18:16Z

I don't have a problem with the idea or the code, but the timing of doing this right now is weird.

As with much of the other stuff I did, the timing depends more on finding the time to do it than on achieving a deadline or a particular general goal.

I've just spent the last 3 days if not longer trying to get out a release ensuring all the parts work together. (And I am not sure they completely do). Something like this now ensures things will be a little bit out of whack again and puts us back in the state of a little off-ness. I don't think some of the code will be able to handle 3.13 unless an older pip is used. And code in other repositories may need additional adjusting too.

The changes here (except for the tests) ensure that mathics-core tests still pass in Python 3.13. This is not a release or something that breaks the rest of the code. The changes do not seem to affect the behavior of the rest of the components, but in six months or a year, when we want to start the next release, they would make the process simpler.

And of course, there are other PRs in the backlog. These change the API which will also require changing some other bits of code outside of this repository.

To accommodate moving forward which we need to do and keep some stability for a little while right now, I suggest we have an "unstable" branch that we merge to, along with the master branch which is compatible across other projects.

That sounds reasonable.

Your thoughts?

rocky · 2025-01-29T15:38:58Z

As with much of the other stuff I did, the timing depends more on finding the time to do it than on achieving a deadline or a particular general goal.

This is a problem that can be solved by working in a more coordinated way.

From the project's perspective, it is better to have a plan for moving forward that we agree on and discuss.

You put in a checklist of ideas for the 8.0.0 release. Some of these have been deferred. Items in that list were picked precisely not to be disruptive.

And we have open issues. I suppose we should add upgrading to 3.13 to that list, but 3.13 was not on that list. Upgrading to a newer Python release, especially with the way Python and its ecosystem move nowadays, is disruptive.

And if you have a question about doing something you could just mention it or ask about it in advance.

mmatera · 2025-01-29T16:23:07Z

As with much of the other stuff I did, the timing depends more on finding the time to do it than on achieving a deadline or a particular general goal.

This is a problem that can be solved by working in a more coordinated way.

From the project's perspective, it is better to have a plan for moving forward that we agree on and discuss.

You put in a checklist of ideas for the 8.0.0 release. Some of these have been deferred. Items in that list were picked precisely not to be disruptive.

And we have open issues. I suppose we should add upgrading to 3.13 to that list, but 3.13 was not on that list. Upgrading to a newer Python release, especially with the way Python and its ecosystem move nowadays, is disruptive.

And if you have a question about doing something you could just mention it or ask about it in advance.

OK, but the changes in this PR are closely related to the work done before the release, relative to adjusting the get_function method and the documentation building. Even for the currently supported versions, these changes make the code more robust. But OK, I will split this in two parts, and leave this for later.

BTW, the support for LaTeX expressions in the online Mathics-Django using MathJax, something that was in the list for the pre-release, is done: Mathics3/mathics-django#221

rocky · 2025-01-29T16:37:56Z

As with much of the other stuff I did, the timing depends more on finding the time to do it than on achieving a deadline or a particular general goal.

This is a problem that can be solved by working in a more coordinated way.
From the project's perspective, it is better to have a plan for moving forward that we agree on and discuss.
You put in a checklist of ideas for the 8.0.0 release. Some of these have been deferred. Items in that list were picked precisely not to be disruptive.
And we have open issues. I suppose we should add upgrading to 3.13 to that list, but 3.13 was not on that list. Upgrading to a newer Python release, especially with the way Python and its ecosystem move nowadays, is disruptive.
And if you have a question about doing something you could just mention it or ask about it in advance.

OK, but the changes in this PR are closely related to the work done before the release, relative to adjusting the get_function method and the documentation building.

You may see it that way, but as the discussion I hope shows, it isn't like that. There could be a lesson here: mention first, code later.

Even for the currently supported versions, these changes make the code more robust.

Again, it is not about whether certain changes are good, but more about coordinating, timing, and appropriateness for the situation right now.

But OK, I will split this in two parts, and leave this for later.

Thank you!

BTW, the support for LaTeX expressions in the online Mathics-Django using MathJax, something that was in the list for the pre-release, is done: Mathics3/mathics-django#221

Sorry, I lost track of this. In my last look, you mentioned that MathJaX needed to be updated. I see that's been done now.

Looking at it and trying now. Thanks for doing this.

As for release, We can always come out with a minor update for the Django portion. And that might be a good thing to do.

The Django portion has this weirdness that we are not using PyPI for CI testing. It might have something to do with where it is expecting JSON tables to be located or that the tables are not getting created. I let this slide in the overall 8.0.0 revision of everything because things were just getting delayed.

mmatera added 3 commits January 27, 2025 23:48

since Python 3.13, get_functions must check if the method is callable

f27d9ea

In Python 3.13 margin spaces in docstrings are trimmed

bcd883a

Python 3.13 converts tabs into simple spaces in docstrings

eb2232e

mmatera changed the title ~~Fixes for 3 13~~ Fixes for compatibility with Python 3.13 Jan 28, 2025

mmatera commented Jan 28, 2025

View reviewed changes

rocky reviewed Jan 28, 2025

View reviewed changes

push workflows

f73364b

mmatera force-pushed the fixes_for_3_13 branch from 9e2a614 to f73364b Compare January 28, 2025 22:21

mmatera added 2 commits January 28, 2025 19:22

Merge branch 'master' into fixes_for_3_13

48070e9

Update windows.yml

29706ae

mmatera changed the base branch from master to unstable January 29, 2025 17:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes for compatibility with Python 3.13 #1321

Fixes for compatibility with Python 3.13 #1321

mmatera commented Jan 28, 2025

mmatera Jan 28, 2025

mmatera Jan 28, 2025

rocky Jan 28, 2025

mmatera Jan 28, 2025

mmatera Jan 28, 2025

mmatera Jan 28, 2025

mmatera Jan 28, 2025

mmatera Jan 28, 2025

rocky commented Jan 28, 2025

mmatera commented Jan 28, 2025

rocky commented Jan 28, 2025 •

edited

Loading

rocky Jan 28, 2025 •

edited

Loading

mmatera Jan 28, 2025

mmatera Jan 28, 2025

rocky Jan 28, 2025 •

edited

Loading

mmatera Jan 28, 2025

rocky commented Jan 29, 2025 •

edited

Loading

rocky commented Jan 29, 2025

mmatera commented Jan 29, 2025

rocky commented Jan 29, 2025 •

edited

Loading

mmatera commented Jan 29, 2025

rocky commented Jan 29, 2025 •

edited

Loading

Fixes for compatibility with Python 3.13 #1321

Are you sure you want to change the base?

Fixes for compatibility with Python 3.13 #1321

Conversation

mmatera commented Jan 28, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rocky commented Jan 28, 2025

mmatera commented Jan 28, 2025

rocky commented Jan 28, 2025 • edited Loading

rocky Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rocky Jan 28, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rocky commented Jan 29, 2025 • edited Loading

rocky commented Jan 29, 2025

mmatera commented Jan 29, 2025

rocky commented Jan 29, 2025 • edited Loading

mmatera commented Jan 29, 2025

rocky commented Jan 29, 2025 • edited Loading

rocky commented Jan 28, 2025 •

edited

Loading

rocky Jan 28, 2025 •

edited

Loading

rocky Jan 28, 2025 •

edited

Loading

rocky commented Jan 29, 2025 •

edited

Loading

rocky commented Jan 29, 2025 •

edited

Loading

rocky commented Jan 29, 2025 •

edited

Loading