Merge pull request #816 from neo4j-contrib/rc/5.3.2

Rc/5.3.2

Author: mariusconjeaud

Changelog

@@ -1,3 +1,7 @@
+Version 5.3.2 2024-06
+* Add support for Vector and Fulltext indexes creation
+* Add DateTimeNeo4jFormatProperty for Neo4j native datetime format
+
 Version 5.3.1 2024-05
 * Add neomodel_generate_diagram script, which generates a graph model diagram based on your neomodel class definitions. Arrows and PlantUML dot options
 * Fix bug in async iterator async for MyClass.nodes

doc/source/configuration.rst

@@ -32,7 +32,7 @@ Adjust driver configuration - these options are only available for this connecti
     config.MAX_TRANSACTION_RETRY_TIME = 30.0  # default
     config.RESOLVER = None  # default
     config.TRUST = neo4j.TRUST_SYSTEM_CA_SIGNED_CERTIFICATES  # default
-    config.USER_AGENT = neomodel/v5.3.1  # default
+    config.USER_AGENT = neomodel/v5.3.2  # default
 
 Setting the database name, if different from the default one::
 

doc/source/extending.rst

@@ -21,6 +21,16 @@ Creating purely abstract classes is achieved using the `__abstract_node__` prope
             self.balance = self.balance + int(amount)
             self.save()
 
+Custom label
+------------
+By default, neomodel uses the class name as the label for nodes. This can be overridden by setting the __label__ property on the class::
+
+    class PersonClass(StructuredNode):
+        __label__ = "Person"
+        name = StringProperty(unique_index=True)
+
+Creating a PersonClass instance and saving it to the database will result in a node with the label "Person".
+
 
 Optional Labels
 ---------------
@@ -131,11 +141,16 @@ Consider for example the following snippet of code::
     class PilotPerson(BasePerson):
         pass
 
+
+    class UserClass(StructuredNode):
+        __label__ = "User"
+
 Once this script is executed, the *node-class registry* would contain the following entries: ::
 
     {"BasePerson"}                    --> class BasePerson
     {"BasePerson", "TechnicalPerson"} --> class TechnicalPerson
     {"BasePerson", "PilotPerson"}     --> class PilotPerson
+    {"User"}                          --> class UserClass
 
 Therefore, a ``Node`` with labels ``"BasePerson", "TechnicalPerson"`` would lead to the instantiation of a
 ``TechnicalPerson`` object. This automatic resolution is **optional** and can be invoked automatically via
@@ -184,12 +199,52 @@ This automatic class resolution however, requires a bit of caution:
           ``{"BasePerson", "PilotPerson"}`` to ``PilotPerson`` **in the global scope** with a mapping of the same
           set of labels but towards the class defined within the **local scope** of ``some_function``.
 
+3. Two classes with different names but the same __label__ override will also result in a ``ClassAlreadyDefined`` exception.
+   This can be avoided under certain circumstances, as explained in the next section on 'Database specific labels'. 
+
 Both ``ModelDefinitionMismatch`` and ``ClassAlreadyDefined`` produce an error message that returns the labels of the
 node that created the problem (either the `Node` returned from the database or the class that was attempted to be
 redefined) as well as the state of the current *node-class registry*. These two pieces of information can be used to
 debug the model mismatch further.
 
 
+Database specific labels
+------------------------
+**Only for Neo4j Enterprise Edition, with multiple databases**
+
+In some cases, it is necessary to have a class with a label that is not unique across the database.
+This can be achieved by setting the `__target_databases__` property to a list of strings ::
+    class PatientOne(AsyncStructuredNode):
+        __label__ = "Patient"
+        __target_databases__ = ["db_one"]
+        name = StringProperty()
+
+    class PatientTwo(AsyncStructuredNode):
+        __label__ = "Patient"
+        __target_databases__ = ["db_two"]
+        identifier = StringProperty()
+
+In this example, both `PatientOne` and `PatientTwo` have the label "Patient", but these will be mapped in a database-specific *node-class registry*.
+
+Now, if you fetch a node with label Patient from your database with auto resolution enabled, neomodel will try to resolve it to the correct class
+based on the database it was fetched from ::
+    db.set_connection("bolt://neo4j:password@localhost:7687/db_one")
+    patients = db.cypher_query("MATCH (n:Patient) RETURN n", resolve_objects=True) --> instance of PatientOne
+
+The following will result in a ``ClassAlreadyDefined`` exception, because when retrieving from ``db_one``,
+neomodel would not be able to decide which model to parse into ::
+    class GeneralPatient(AsyncStructuredNode):
+        __label__ = "Patient"
+        name = StringProperty()
+
+    class PatientOne(AsyncStructuredNode):
+        __label__ = "Patient"
+        __target_databases__ = ["db_one"]
+        name = StringProperty()
+
+.. warning:: This does not prevent you from saving a node to the "wrong database". So you can still save an instance of PatientTwo to database "db_one".
+
+
 ``neomodel`` under multiple processes and threads
 -------------------------------------------------
 It is very important to realise that neomodel preserves a mapping of the set of labels associated with the Neo4J

doc/source/getting_started.rst

@@ -73,6 +73,8 @@ in the case of ``Relationship`` it will be possible to be queried in either dire
 Neomodel automatically creates a label for each ``StructuredNode`` class in the database with the corresponding indexes
 and constraints.
 
+.. _inspect_database_doc:
+
 Database Inspection - Requires APOC
 ===================================
 You can inspect an existing Neo4j database to generate a neomodel definition file using the ``inspect`` command::
@@ -239,7 +241,7 @@ additional relations with a single call::
 
     # The following call will generate one MATCH with traversal per
     # item in .fetch_relations() call
-    results = Person.nodes.all().fetch_relations('country')
+    results = Person.nodes.fetch_relations('country').all()
     for result in results:
         print(result[0]) # Person
         print(result[1]) # associated Country
@@ -248,33 +250,38 @@ You can traverse more than one hop in your relations using the
 following syntax::
 
     # Go from person to City then Country
-    Person.nodes.all().fetch_relations('city__country')
+    Person.nodes.fetch_relations('city__country').all()
 
 You can also force the use of an ``OPTIONAL MATCH`` statement using
 the following syntax::
 
     from neomodel.match import Optional
 
-    results = Person.nodes.all().fetch_relations(Optional('country'))
+    results = Person.nodes.fetch_relations(Optional('country')).all()
+
+.. note::
+
+    Any relationship that you intend to traverse using this method **MUST have a model defined**, even if only the default StructuredRel, like::
+        
+        class Person(StructuredNode):
+            country = RelationshipTo(Country, 'IS_FROM', model=StructuredRel)
+
+    Otherwise, neomodel will not be able to determine which relationship model to resolve into, and will fail.
 
 .. note::
 
    You can fetch one or more relations within the same call
    to `.fetch_relations()` and you can mix optional and non-optional
    relations, like::
 
-    Person.nodes.all().fetch_relations('city__country', Optional('country'))
+    Person.nodes.fetch_relations('city__country', Optional('country')).all()
 
 .. note::
 
-   This feature is still a work in progress for extending path traversal and fecthing.
-   It currently stops at returning the resolved objects as they are returned in Cypher.
-   So for instance, if your path looks like ``(startNode:Person)-[r1]->(middleNode:City)<-[r2]-(endNode:Country)``,
+   If your path looks like ``(startNode:Person)-[r1]->(middleNode:City)<-[r2]-(endNode:Country)``,
    then you will get a list of results, where each result is a list of ``(startNode, r1, middleNode, r2, endNode)``.
    These will be resolved by neomodel, so ``startNode`` will be a ``Person`` class as defined in neomodel for example.
 
-   If you want to go further in the resolution process, you have to develop your own parser (for now).
-
 
 Async neomodel
 ==============

doc/source/index.rst

@@ -73,6 +73,7 @@ Contents
    relationships
    properties
    spatial_properties
+   schema_management
    queries
    cypher
    transactions

doc/source/properties.rst

@@ -6,15 +6,15 @@ Property types
 
 The following properties are available on nodes and relationships:
 
-====================================================  ===========================================================
-:class:`~neomodel.properties.AliasProperty`           :class:`~neomodel.properties.IntegerProperty`
-:class:`~neomodel.properties.ArrayProperty`           :class:`~neomodel.properties.JSONProperty`
-:class:`~neomodel.properties.BooleanProperty`         :class:`~neomodel.properties.RegexProperty`
-:class:`~neomodel.properties.DateProperty`            :class:`~neomodel.properties.StringProperty` (:ref:`Notes <properties_notes>`)
-:class:`~neomodel.properties.DateTimeProperty`        :class:`~neomodel.properties.UniqueIdProperty`
-:class:`~neomodel.properties.DateTimeFormatProperty`  :class:`~neomodel.contrib.spatial_properties.PointProperty`
-:class:`~neomodel.properties.FloatProperty`           \
-====================================================  ===========================================================
+=========================================================  ===========================================================
+:class:`~neomodel.properties.AliasProperty`                :class:`~neomodel.properties.FloatProperty`   
+:class:`~neomodel.properties.ArrayProperty`                :class:`~neomodel.properties.IntegerProperty`
+:class:`~neomodel.properties.BooleanProperty`              :class:`~neomodel.properties.JSONProperty`
+:class:`~neomodel.properties.DateProperty`                 :class:`~neomodel.properties.RegexProperty`
+:class:`~neomodel.properties.DateTimeProperty`             :class:`~neomodel.properties.StringProperty` (:ref:`Notes <properties_notes>`)
+:class:`~neomodel.properties.DateTimeFormatProperty`       :class:`~neomodel.properties.UniqueIdProperty` 
+:class:`~neomodel.properties.DateTimeNeo4jFormatProperty`  :class:`~neomodel.contrib.spatial_properties.PointProperty`
+=========================================================  ===========================================================
 
 
 Naming Convention

doc/source/schema_management.rst

@@ -0,0 +1,93 @@
+=================
+Schema management
+=================
+
+Neo4j allows a flexible schema management, where you can define indexes and constraints on the properties of nodes and relationships.
+To learn more, please refer to the `Neo4j schema documentation <https://neo4j.com/docs/getting-started/cypher-intro/schema/>`_.
+
+Defining your model
+-------------------
+
+neomodel allows you to define indexes and constraints in your node and relationship classes, like so: ::
+
+    from neomodel import (StructuredNode, StructuredRel, StringProperty,
+        IntegerProperty, RelationshipTo)
+        
+    class LocatedIn(StructuredRel):
+        since = IntegerProperty(index=True)
+
+    class Country(StructuredNode):
+        code = StringProperty(unique_index=True)
+
+    class City(StructuredNode):
+        name = StringProperty(index=True)
+        country = RelationshipTo(Country, 'FROM_COUNTRY', model=LocatedIn)
+
+
+Applying constraints and indexes
+--------------------------------
+After creating your model, any constraints or indexes must be applied to Neo4j and ``neomodel`` provides a
+script (:ref:`neomodel_install_labels`) to automate this: ::
+
+    $ neomodel_install_labels yourapp.py someapp.models --db bolt://neo4j_username:neo4j_password@localhost:7687
+
+It is important to execute this after altering the schema and observe the number of classes it reports.
+
+Ommitting the ``--db`` argument will default to the ``NEO4J_BOLT_URL`` environment variable. This is useful for masking
+your credentials.
+
+.. note::
+    The script will only create indexes and constraints that are defined in your model classes. It will not remove any
+    existing indexes or constraints that are not defined in your model classes.
+
+Indexes
+=======
+
+The following indexes are supported:
+
+- ``index=True``: This will create the default Neo4j index on the property (currently RANGE).
+- ``fulltext_index=FulltextIndex()``: This will create a FULLTEXT index on the property. Only available for Neo4j version 5.16 or higher. With this one, you can define the following options:
+    - ``analyzer``: The analyzer to use. The default is ``standard-no-stop-words``.
+    - ``eventually_consistent``: Whether the index should be eventually consistent. The default is ``False``.
+  
+Please refer to the `Neo4j documentation <https://neo4j.com/docs/cypher-manual/current/indexes/semantic-indexes/full-text-indexes/#configuration-settings>`_. for more information on fulltext indexes.
+
+- ``vector_index=VectorIndex()``: This will create a VECTOR index on the property. Only available for Neo4j version 5.15 (node) and 5.18 (relationship) or higher. With this one, you can define the following options:
+    - ``dimensions``: The dimension of the vector. The default is 1536.
+    - ``similarity_function``: The similarity algorithm to use. The default is ``cosine``.
+
+Those indexes are available for both node- and relationship properties.
+
+.. note:: 
+    Yes, you can create multiple indexes of a different type on the same property. For example, a default index and a fulltext index.
+
+.. note:: 
+    For the semantic indexes (fulltext and vector), this allows you to create indexes, but searching those indexes require using Cypher queries.
+    This is because Cypher only supports querying those indexes through a specific procedure for now.
+
+Full example: ::
+
+    from neomodel import StructuredNode, StringProperty, FulltextIndex, VectorIndex
+    class VeryIndexedNode(StructuredNode):
+        name = StringProperty(
+            index=True,
+            fulltext_index=FulltextIndex(analyzer='english', eventually_consistent=True)
+            vector_index=VectorIndex(dimensions=512, similarity_function='euclidean')
+        )
+
+Constraints
+===========
+
+The following constraints are supported:
+
+- ``unique_index=True``: This will create a uniqueness constraint on the property. Available for both nodes and relationships (Neo4j version 5.7 or higher).
+
+.. note::
+    The uniquess constraint of Neo4j is not supported as such, but using ``required=True`` on a property serves the same purpose.
+
+
+Extracting the schema from a database
+=====================================
+
+You can extract the schema from an existing database using the ``neomodel_inspect_database`` script (:ref:`inspect_database_doc`).
+This script will output the schema in the neomodel format, including indexes and constraints.

neomodel/__init__.py

@@ -25,15 +25,18 @@
     BooleanProperty,
     DateProperty,
     DateTimeFormatProperty,
+    DateTimeNeo4jFormatProperty,
     DateTimeProperty,
     EmailProperty,
     FloatProperty,
+    FulltextIndex,
     IntegerProperty,
     JSONProperty,
     NormalizedProperty,
     RegexProperty,
     StringProperty,
     UniqueIdProperty,
+    VectorIndex,
 )
 from neomodel.sync_.cardinality import One, OneOrMore, ZeroOrMore, ZeroOrOne
 from neomodel.sync_.core import (

neomodel/_version.py

@@ -1 +1 @@
-__version__ = "5.3.1"
+__version__ = "5.3.2"