Merge pull request #419 from mpsonntag/updateRDFlib

rdflib dependency update, preparation release v1.5.3

Author: achilleas-k

.github/workflows/run-tests.yml

@@ -13,7 +13,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest]
-        python-version: ["3.8", "3.9", "3.10"]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
     steps:
       - uses: actions/checkout@v2
       - name: Setup Python ${{ matrix.python-version }}
@@ -35,7 +35,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-20.04]
-        python-version: ["3.6", "3.7"]
+        python-version: ["3.7"]
     steps:
       - uses: actions/checkout@v2
       - name: Setup Python ${{ matrix.python-version }}
@@ -61,7 +61,7 @@ jobs:
         # and fails on install. MacOS Python version 3.9 build will stay
         # disabled until this is resolved.
         #python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]
-        python-version: ["3.6", "3.7", "3.8", "3.10"]
+        python-version: ["3.7", "3.8", "3.10"]
     steps:
       - uses: actions/checkout@v2
       - name: Setup Python ${{ matrix.python-version }}

CHANGELOG.md

@@ -4,6 +4,24 @@ Used to document all changes from previous releases and collect changes
 until the next release.
 
 
+# Version 1.5.3
+
+## Support rdflib version 6+
+
+The rdflib upgrade from version 5.0.0 to 6+ introduced breaking changes 
+in the rdflib library. This update fixes these breaks without changing 
+functionality, unpins the rdflib version and defaults to the latest
+version on install.
+
+## Dropping official support for Python 3.5 and 3.6
+
+The introduction of the rdflib 6+ library restricts the
+Python support for 3.7+.
+
+It is still possible to install odml with a Python version <=3.6, 
+but these versions are no longer tested and officially supported.
+
+
 # Version 1.5.2
 
 ## Pinning rdflib version to 5.0.0 until further notice

README.md

@@ -77,7 +77,7 @@ release notes](https://github.com/G-Node/python-odml/releases).
 
 # Dependencies
 
-* Python 3.6+
+* Python 3.7+
 * Python packages:
 
   * lxml (version 3.7.2)
@@ -101,7 +101,9 @@ release notes](https://github.com/G-Node/python-odml/releases).
 
 ## Previous Python versions
 
-Python 2 has reached end of life. We will not keep any future versions of odml Python 2 compatible and will completely drop support for Python 2 with August 2020. We also recommend using a Python version >= 3.6. If a Python version < 3.6 is a requirement, the following dependency needs to be installed as well:
+Python 2 has reached end of life. Current and future versions of odml are not Python 2 compatible. We removed support 
+for Python 2 in August 2020 with version 1.5.2. We also recommend using a Python version >= 3.7. If a 
+Python version < 3.7 is a requirement, the following dependency needs to be installed as well:
 
 * pip install
   * enum34 (version 0.4.4)

appveyor.yml

@@ -4,9 +4,6 @@ image: Visual Studio 2017
 
 environment:
   matrix:
-    - PYTHON: "C:\\Python36"
-      PYVER: 3
-      BITS: 32
     - PYTHON: "C:\\Python37"
       PYVER: 3
       BITS: 32
@@ -21,9 +18,10 @@ environment:
       APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
       PYVER: 3
       BITS: 32
-    - PYTHON: "C:\\Python36-x64"
+    - PYTHON: "C:\\Python311"
+      APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
       PYVER: 3
-      BITS: 64
+      BITS: 32
     - PYTHON: "C:\\Python37-x64"
       PYVER: 3
       BITS: 64
@@ -38,6 +36,10 @@ environment:
       APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
       PYVER: 3
       BITS: 64
+    - PYTHON: "C:\\Python311-x64"
+      APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
+      PYVER: 3
+      BITS: 64
 
 init:
   - "ECHO %PYTHON% %vcvars% (%bits%)"

doc/support-classes.rst

@@ -7,7 +7,7 @@ These classes are
 
 
 Validation
--------
+----------
 .. autoclass:: odml.validation.Validation
    :members:
    :inherited-members:

doc/tutorial.rst

@@ -1,13 +1,12 @@
-
 =============
 odML Tutorial
 =============
 
 :Author:
-    Lyuba Zehl;
+    Lyuba Zehl, Michael Sonntag;
     based on work by Hagen Fritsch
 :Release:
-    1.4
+    1.5
 :License:
     Creative Commons Attribution-ShareAlike 4.0 International
     `License <http://creativecommons.org/licenses/by-sa/4.0/>`_
@@ -89,7 +88,7 @@ the project name `python-odml <https://github.com/G-Node/python-odml>`_.
 Dependencies
 ------------
 
-The Python-odML library (version 1.4+) runs under Python 3.6+.
+The Python-odML library (version 1.5+) is tested and fully supported using Python 3.7+.
 
 Additionally, the Python-odML library depends on the lxml, pyyaml and rdflib python packages.
 
@@ -102,7 +101,9 @@ On Ubuntu, the dependency packages are available as ``python-lxml``, ``python-ya
 Note that on Ubuntu 14.04, the latter package additionally requires the
 installation of ``libxml2-dev``, ``libxslt1-dev``, and ``lib32z1-dev``.
 
-Python 2 has reached end of life. We will not keep any future versions of odml Python 2 compatible and also recommend using a Python version >= 3.6. If a Python version < 3.6 is a requirement, the following dependency needs to be installed as well:
+Python 2 has reached end of life. Current and future versions of odml are not Python 2 compatible. We removed support
+for Python 2 in August 2020 with version 1.5.2. We also recommend using a Python version >= 3.7. If a
+Python version < 3.7 is a requirement, the following dependency needs to be installed as well:
 
 The ``enum34`` package with a ``pip`` installation or ``python-enum`` using the OS package manager.
 

odml/__init__.py

@@ -33,7 +33,7 @@ def _format_warning(warn_msg, *args, **kwargs):
     msg = "Python 2 has reached end of live."
     msg += "\n\todML support for Python 2 has been dropped."
     warnings.warn(msg, category=DeprecationWarning, stacklevel=2)
-elif _python_version.major == 3 and _python_version.minor < 6:
+elif _python_version.major == 3 and _python_version.minor < 7:
     msg = "The '%s' package is not tested with your Python version. " % __name__
     msg += "\n\tPlease consider upgrading to the latest Python distribution."
     warnings.warn(msg)

odml/info.json

@@ -1,5 +1,5 @@
 {
-  "VERSION": "1.5.2",
+  "VERSION": "1.5.3",
   "FORMAT_VERSION": "1.1",
   "AUTHOR": "Hagen Fritsch, Jan Grewe, Christian Kellner, Achilleas Koutsou, Michael Sonntag, Lyuba Zehl",
   "COPYRIGHT": "(c) 2011-2023, German Neuroinformatics Node",
@@ -8,10 +8,10 @@
   "CLASSIFIERS": [
     "Development Status :: 5 - Production/Stable",
     "Programming Language :: Python",
-    "Programming Language :: Python :: 3.6",
     "Programming Language :: Python :: 3.7",
     "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
     "Topic :: Scientific/Engineering",
     "Intended Audience :: Science/Research",
     "License :: OSI Approved :: BSD License"

odml/tools/rdf_converter.py

@@ -9,8 +9,15 @@
 import warnings
 
 from io import StringIO
+
+from rdflib import __version__ as rdflib_version
 from rdflib import Graph, Literal, URIRef
 from rdflib.graph import Seq
+try:
+    from rdflib.container import Seq as CollSeq
+except ImportError as exc:
+    # annoy people to upgrade their rdflib version but still support the usage
+    print("deprecated rdflib version. Please upgrade to the latest version.")
 from rdflib.namespace import XSD, RDF, RDFS
 
 import yaml
@@ -25,6 +32,14 @@
 ODML_NS = Format.namespace()
 
 
+def rdflib_version_major():
+    version_split = rdflib_version.split(".")
+    if len(version_split) < 3 or not version_split[0].isdigit():
+        print("Could not parse rdflib version %s" % rdflib_version)
+        return 0
+    return int(version_split[0])
+
+
 def load_rdf_subclasses():
     """
     load_rdf_subclasses loads odml section types to RDF Section subclass types
@@ -133,16 +148,23 @@ def save_odml_values(self, parent_node, rdf_predicate, values):
         # Once rdflib upgrades this should be reversed to RDF:li again!
         # see https://github.com/RDFLib/rdflib/issues/280
         # -- keep until supported
-        # bag = URIRef(ODML_NS + str(uuid.uuid4()))
-        # self.graph.add((bag, RDF.type, RDF.Bag))
-        # self.graph.add((curr_node, fmt.rdf_map(k), bag))
-        # for curr_val in values:
-        #     self.graph.add((bag, RDF.li, Literal(curr_val)))
-        counter = 1
-        for curr_val in values:
-            custom_predicate = "%s_%s" % (str(RDF), counter)
-            self.graph.add((seq, URIRef(custom_predicate), Literal(curr_val)))
-            counter = counter + 1
+        #bag = URIRef(ODML_NS + str(uuid.uuid4()))
+        #self.graph.add((bag, RDF.type, RDF.Bag))
+        #self.graph.add((parent_node, rdf_predicate, bag))
+        #for curr_val in values:
+        #    self.graph.add((bag, RDF.li, Literal(curr_val)))
+        if rdflib_version_major() >= 6:
+            seq_list = []
+            for curr_val in values:
+                seq_list.append(Literal(curr_val))
+            _ = CollSeq(self.graph, seq, seq_list)
+        else:
+            # manually create and use the value blank nodes order
+            counter = 1
+            for curr_val in values:
+                custom_predicate = "%s_%s" % (str(RDF), counter)
+                self.graph.add((seq, URIRef(custom_predicate), Literal(curr_val)))
+                counter = counter + 1
 
     def save_odml_list(self, parent_node, rdf_predicate, odml_list):
         """
@@ -349,9 +371,13 @@ def _parse_custom_subclasses(self, custom_subclasses):
             self.section_subclasses[k] = val
 
     def __str__(self):
+        if rdflib_version_major() >= 6:
+            return self.convert_to_rdf().serialize(format='turtle')
         return self.convert_to_rdf().serialize(format='turtle').decode("utf-8")
 
     def __unicode__(self):
+        if rdflib_version_major() >= 6:
+            return self.convert_to_rdf().serialize(format='turtle')
         return self.convert_to_rdf().serialize(format='turtle').decode("utf-8")
 
     def get_rdf_str(self, rdf_format="turtle"):
@@ -370,6 +396,8 @@ def get_rdf_str(self, rdf_format="turtle"):
             msg = "%s Please choose from the list: %s" % (msg, list(RDF_CONVERSION_FORMATS))
             raise ValueError(msg)
 
+        if rdflib_version_major() >= 6:
+            return self.convert_to_rdf().serialize(format=rdf_format)
         return self.convert_to_rdf().serialize(format=rdf_format).decode("utf-8")
 
     def write_file(self, filename, rdf_format="turtle"):
@@ -516,18 +544,26 @@ def parse_property(self, prop_uri):
             if attr[0] == "value" and elems:
                 prop_attrs[attr[0]] = []
 
-                # rdflib does not respect order with RDF.li items yet, see comment above
-                # support both RDF.li and rdf:_nnn for now.
-                # Remove rdf:_nnn once rdflib respects RDF.li order in an RDF.Seq obj.
-                values = list(self.graph.objects(subject=elems[0], predicate=RDF.li))
-                if values:
-                    for curr_val in values:
-                        prop_attrs[attr[0]].append(curr_val.toPython())
-                else:
-                    # rdf:__nnn part
+                if rdflib_version_major() >= 6:
+                    # rdflib version 6.x.x+ should support rdf:_nnn only, RDF.li
+                    # are not supported; reverse import the blank node values;
+                    # hopefully in the correct order.
                     val_seq = Seq(graph=self.graph, subject=elems[0])
                     for seq_item in val_seq:
                         prop_attrs[attr[0]].append(seq_item.toPython())
+                else:
+                    # rdflib does not respect order with RDF.li items yet, see comment above
+                    # support both RDF.li and rdf:_nnn for now.
+                    # Remove rdf:_nnn once rdflib respects RDF.li order in an RDF.Seq obj.
+                    values = list(self.graph.objects(subject=elems[0], predicate=RDF.li))
+                    if values:
+                        for curr_val in values:
+                            prop_attrs[attr[0]].append(curr_val.toPython())
+                    else:
+                        # rdf:__nnn part
+                        val_seq = Seq(graph=self.graph, subject=elems[0])
+                        for seq_item in val_seq:
+                            prop_attrs[attr[0]].append(seq_item.toPython())
 
             elif attr[0] == "id":
                 prop_attrs[attr[0]] = prop_uri.split("#", 1)[1]

requirements-test.txt

@@ -1,3 +1,3 @@
+owlrl
 pytest
-owlrl==5.2.3
 requests

scripts/release_tests/run_pypi_odml.sh

@@ -3,7 +3,7 @@
 echo
 echo "-- MAKE SURE TO RUN THIS SCRIPT IN INTERACTIVE MODE '-i' --"
 
-PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|")
+PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|3.11|")
 
 if [[ $# != 1 ]]; then
     echo

scripts/release_tests/run_test_local_odml.sh

@@ -3,7 +3,7 @@
 echo
 echo "-- MAKE SURE TO RUN THIS SCRIPT IN INTERACTIVE MODE '-i' --"
 
-PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|")
+PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|3.11|")
 
 if [[ $# != 1 ]]; then
     echo

scripts/release_tests/run_test_matrix.sh

@@ -99,6 +99,9 @@ function run_script () {
     fi
 }
 
+PYVER=3.11
+run_script
+
 PYVER=3.10
 run_script
 

scripts/release_tests/run_test_pypi_nixodmlconverter.sh

@@ -3,7 +3,7 @@
 echo
 echo "-- MAKE SURE TO RUN THIS SCRIPT IN INTERACTIVE MODE '-i' --"
 
-PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|")
+PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|3.11|")
 
 if [[ $# != 1 ]]; then
     echo

scripts/release_tests/run_test_pypi_odml.sh

@@ -3,7 +3,7 @@
 echo
 echo "-- MAKE SURE TO RUN THIS SCRIPT IN INTERACTIVE MODE '-i' --"
 
-PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|")
+PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|3.11|")
 
 if [[ $# != 1 ]]; then
     echo

scripts/release_tests/run_test_pypi_odmltools.sh

@@ -3,7 +3,7 @@
 echo
 echo "-- MAKE SURE TO RUN THIS SCRIPT IN INTERACTIVE MODE '-i' --"
 
-PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|")
+PY_VER_ARRAY=("|3.5|3.6|3.7|3.8|3.9|3.10|3.11|")
 
 if [[ $# != 1 ]]; then
     echo

setup.py

@@ -31,13 +31,18 @@
 with open('README.md') as f:
     description_text = f.read()
 
-# pyparsing needs to be pinned to 2.4.7 for the time being. setup install fetches a pre-release
-# package that currently results in issues with the rdflib library.
-install_req = ["lxml", "pyyaml>=5.1", "rdflib==5.0.0", "docopt", "pathlib", "pyparsing==2.4.7"]
+install_req = ["docopt", "lxml", "pathlib", "pyyaml>=5.1", "rdflib>=6.0.0"]
+# owlrl depends on rdflib; update any changes in requirements-test.txt as well.
+tests_req = ["owlrl", "pytest", "requests"]
 
-# owlrl depends on rdflib - the pinned version should be removed once the version pin has also been
-# removed from rdflib. Also needs to be updated in requirements-test.txt
-tests_req = ["pytest", "owlrl==5.2.3", "requests"]
+# Keep support for for Python versions below 3.7; relevant for the
+# rdflib usage; rdflib >= 6 does not support Python versions below 3.7.
+if _python_version.minor <= 6:
+    # pyparsing needs to be pinned to 2.4.7 due to issues with the rdflib 5.0.0 library.
+    install_req = ["docopt", "lxml", "pathlib", "pyyaml>=5.1", "rdflib==5.0.0", "pyparsing==2.4.7"]
+
+    # owlrl depends on rdflib and needs to be pinned to a corresponding version.
+    tests_req = ["owlrl==5.2.3", "pytest", "requests"]
 
 setup(
     name='odML',
@@ -66,7 +71,7 @@
     msg = "Python 2 has reached end of live."
     msg += "\n\todML support for Python 2 has been dropped."
     print(msg)
-elif _python_version.major == 3 and _python_version.minor < 6:
+elif _python_version.major == 3 and _python_version.minor < 7:
     msg = "\n\nThis package is not tested with your Python version. "
     msg += "\n\tPlease consider upgrading to the latest Python distribution."
     print(msg)