Skip to content

Graphql refactor + rolling/expanding #2090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 45 commits into from
May 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
5e2c20a
initial graph and node version
Pometry-Team May 8, 2025
e9829e0
Added rolling to everything else
Pometry-Team May 9, 2025
4096926
swapped graphs to be WindowSet
Pometry-Team May 10, 2025
3bf6d06
Add busy loop
Pometry-Team May 12, 2025
2869251
remove _id functions as strings now work for both
Pometry-Team May 12, 2025
03cde18
Renamed graphql objects to remove Gql
Pometry-Team May 12, 2025
62ea023
remove spawn blocking
Pometry-Team May 12, 2025
266463a
tokio spawn edge
Pometry-Team May 12, 2025
9290f48
par edges
Pometry-Team May 13, 2025
6f7de11
Merge branch 'master' into feature/rolling_graphql
Pometry-Team May 13, 2025
194bcb5
fix graph bug and tidy tests
Pometry-Team May 13, 2025
0433e4c
par graph
Pometry-Team May 13, 2025
3b5ced5
remove GqlGraphs
Pometry-Team May 13, 2025
8900d5f
par metagraph
Pometry-Team May 13, 2025
96508f4
partial mutable graph, to be discussed
Pometry-Team May 13, 2025
fae3fa9
par namespace
Pometry-Team May 13, 2025
d379540
Par node
Pometry-Team May 14, 2025
0889d4a
par nodes
Pometry-Team May 14, 2025
1462120
par path
Pometry-Team May 14, 2025
5126fb7
par window set
Pometry-Team May 14, 2025
5552f50
Par props
Pometry-Team May 15, 2025
75a5501
Fixed tests and namespaces
Pometry-Team May 16, 2025
93529c3
fixed test
Pometry-Team May 20, 2025
bb601bb
mutable components
Pometry-Team May 20, 2025
6fc8eee
Fix other test
Pometry-Team May 20, 2025
679768a
Merge branch 'master' into feature/rolling_graphql
miratepuffin May 20, 2025
f07bbd8
Swapped to enums
Pometry-Team May 20, 2025
f194293
Fix recursive write of indexes
Pometry-Team May 21, 2025
8f981b3
fmt
Pometry-Team May 21, 2025
be9e990
fixed step size o
Pometry-Team May 21, 2025
fcd9200
added namespaces and metagraphs
Pometry-Team May 22, 2025
fb89d65
fixed namespace testing
Pometry-Team May 22, 2025
3a0dd14
turn off test_starts
Pometry-Team May 22, 2025
e134f8f
updated graphql
Pometry-Team May 22, 2025
b8de33a
Merge branch 'master' into feature/rolling_graphql
Pometry-Team May 22, 2025
d583daa
fmt
Pometry-Team May 22, 2025
e815ff7
fixed tests
Pometry-Team May 24, 2025
46ffde0
Found bug in rolling where we would get an empty window if the step w…
Pometry-Team May 24, 2025
1e39482
adding tests for graph page and count
Pometry-Team May 24, 2025
3b82d46
fmt
Pometry-Team May 25, 2025
7698850
Added tests for all rolling and expanding
Pometry-Team May 25, 2025
6702797
export_to with spawn_blocking
ljeub-pometry May 27, 2025
36afdaf
initial review fixes
Pometry-Team May 27, 2025
7777a91
Finished rest of tidy
Pometry-Team May 27, 2025
cf522b0
removed some import by accident
Pometry-Team May 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ debug = 0
pometry-storage = { version = ">=0.8.1", path = "pometry-storage" }
#[private-storage]
# pometry-storage = { path = "pometry-storage-private", package = "pometry-storage-private" }
async-graphql = { version = "7.0.13", features = ["dynamic-schema"] }
async-graphql = { version = "7.0.16", features = ["dynamic-schema"] }
bincode = "1.3.3"
async-graphql-poem = "7.0.13"
async-graphql-poem = "7.0.16"
dynamic-graphql = "0.10.1"
reqwest = { version = "0.12.8", default-features = false, features = [
"rustls-tls",
Expand Down
98 changes: 31 additions & 67 deletions python/python/raphtory/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,14 @@ class GraphView(object):
def create_index(self):
"""Create graph index"""

def create_index_in_ram(self):
"""
Creates a graph index in memory (RAM).

This is primarily intended for use in tests and should not be used in production environments,
as the index will not be persisted to disk.
"""

def default_layer(self) -> GraphView:
"""
Return a view of GraphView containing only the default edge layer
Expand Down Expand Up @@ -257,18 +265,6 @@ class GraphView(object):
GraphView: The filtered view
"""

def filter_exploded_edges(self, filter: PropertyFilter) -> GraphView:
"""
Return a filtered view that only includes exploded edges that satisfy the filter

Arguments:
filter (PropertyFilter): The filter to apply to the exploded edge properties. Construct a
filter using `Prop`.

Returns:
GraphView: The filtered view
"""

def filter_nodes(self, filter: PropertyFilter) -> GraphView:
"""
Return a filtered view that only includes nodes that satisfy the filter
Expand Down Expand Up @@ -2323,18 +2319,6 @@ class Node(object):
Node: The filtered view
"""

def filter_exploded_edges(self, filter: PropertyFilter) -> Node:
"""
Return a filtered view that only includes exploded edges that satisfy the filter

Arguments:
filter (PropertyFilter): The filter to apply to the exploded edge properties. Construct a
filter using `Prop`.

Returns:
Node: The filtered view
"""

def filter_nodes(self, filter: PropertyFilter) -> Node:
"""
Return a filtered view that only includes nodes that satisfy the filter
Expand Down Expand Up @@ -2865,18 +2849,6 @@ class Nodes(object):
Nodes: The filtered view
"""

def filter_exploded_edges(self, filter: PropertyFilter) -> Nodes:
"""
Return a filtered view that only includes exploded edges that satisfy the filter

Arguments:
filter (PropertyFilter): The filter to apply to the exploded edge properties. Construct a
filter using `Prop`.

Returns:
Nodes: The filtered view
"""

def filter_nodes(self, filter: PropertyFilter) -> Nodes:
"""
Return a filtered view that only includes nodes that satisfy the filter
Expand Down Expand Up @@ -3387,18 +3359,6 @@ class PathFromNode(object):
PathFromNode: The filtered view
"""

def filter_exploded_edges(self, filter: PropertyFilter) -> PathFromNode:
"""
Return a filtered view that only includes exploded edges that satisfy the filter

Arguments:
filter (PropertyFilter): The filter to apply to the exploded edge properties. Construct a
filter using `Prop`.

Returns:
PathFromNode: The filtered view
"""

def filter_nodes(self, filter: PropertyFilter) -> PathFromNode:
"""
Return a filtered view that only includes nodes that satisfy the filter
Expand Down Expand Up @@ -3832,18 +3792,6 @@ class PathFromGraph(object):
PathFromGraph: The filtered view
"""

def filter_exploded_edges(self, filter: PropertyFilter) -> PathFromGraph:
"""
Return a filtered view that only includes exploded edges that satisfy the filter

Arguments:
filter (PropertyFilter): The filter to apply to the exploded edge properties. Construct a
filter using `Prop`.

Returns:
PathFromGraph: The filtered view
"""

def filter_nodes(self, filter: PropertyFilter) -> PathFromGraph:
"""
Return a filtered view that only includes nodes that satisfy the filter
Expand Down Expand Up @@ -6029,7 +5977,15 @@ class Prop(object):
def __new__(cls, name: str) -> Prop:
"""Create and return a new object. See help(type) for accurate signature."""

def any(self, values: set[PropValue]) -> PropertyFilter:
def contains(self, value) -> PropertyFilter:
"""
Create a filter that keeps entities that contains the property

Returns:
PropertyFilter: the property filter
"""

def is_in(self, values: set[PropValue]) -> PropertyFilter:
"""
Create a filter that keeps entities if their property value is in the set

Expand All @@ -6048,6 +6004,18 @@ class Prop(object):
PropertyFilter: the property filter
"""

def is_not_in(self, values: set[PropValue]) -> PropertyFilter:
"""
Create a filter that keeps entities if their property value is not in the set or
if they don't have the property

Arguments:
values (set[PropValue]): the set of values to exclude

Returns:
PropertyFilter: the property filter
"""

def is_some(self) -> PropertyFilter:
"""
Create a filter that only keeps entities if they have the property
Expand All @@ -6056,13 +6024,9 @@ class Prop(object):
PropertyFilter: the property filter
"""

def not_any(self, values: set[PropValue]) -> PropertyFilter:
def not_contains(self, value) -> PropertyFilter:
"""
Create a filter that keeps entities if their property value is not in the set or
if they don't have the property

Arguments:
values (set[PropValue]): the set of values to exclude
Create a filter that keeps entities that do not contain the property

Returns:
PropertyFilter: the property filter
Expand Down
2 changes: 2 additions & 0 deletions python/python/raphtory/filter/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,5 @@ from pandas import DataFrame
from os import PathLike
import networkx as nx # type: ignore
import pyvis # type: ignore

def Property(name): ...
88 changes: 88 additions & 0 deletions python/python/raphtory/node_state/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -6847,3 +6847,91 @@ class NodeLayout(object):
Returns:
Iterator[list[float]]: Iterator over values
"""

class NodeStateF64String(object):
def __eq__(self, value):
"""Return self==value."""

def __ge__(self, value):
"""Return self>=value."""

def __getitem__(self, key):
"""Return self[key]."""

def __gt__(self, value):
"""Return self>value."""

def __iter__(self):
"""Implement iter(self)."""

def __le__(self, value):
"""Return self<=value."""

def __len__(self):
"""Return len(self)."""

def __lt__(self, value):
"""Return self<value."""

def __ne__(self, value):
"""Return self!=value."""

def __repr__(self):
"""Return repr(self)."""

def get(
self, node: NodeInput, default: Optional[Tuple[float, str]] = None
) -> Optional[Tuple[float, str]]:
"""
Get value for node

Arguments:
node (NodeInput): the node
default (Optional[Tuple[float, str]]): the default value. Defaults to None.

Returns:
Optional[Tuple[float, str]]: the value for the node or the default value
"""

def items(self) -> Iterator[Tuple[Node, Tuple[float, str]]]:
"""
Iterate over items

Returns:
Iterator[Tuple[Node, Tuple[float, str]]]: Iterator over items
"""

def nodes(self) -> Nodes:
"""
Iterate over nodes

Returns:
Nodes: The nodes
"""

def sorted_by_id(self) -> NodeStateF64String:
"""
Sort results by node id

Returns:
NodeStateF64String: The sorted node state
"""

def to_df(self) -> DataFrame:
"""
Convert results to pandas DataFrame

The DataFrame has two columns, "node" with the node ids and "value" with
the corresponding values.

Returns:
DataFrame: the pandas DataFrame
"""

def values(self) -> Iterator[Tuple[float, str]]:
"""
Iterate over values

Returns:
Iterator[Tuple[float, str]]: Iterator over values
"""
42 changes: 24 additions & 18 deletions python/test_utils/filters_setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,28 @@ def func(graph):
for initializer in initializers:
graph = initializer(graph)
return graph

return func


def init_graph(graph):
nodes = [
(1, 1, {"p1": "shivam_kapoor", "p9": 5, "p10": "Paper_airplane"}, "fire_nation"),
(2, 2, {"p1": "prop12", "p2": 2, "p10": "Paper_ship"}, "air_nomads"),
(3, 1, {"p1": "shivam_kapoor", "p9": 5}, "fire_nation"),
(3, 3, {"p2": 6, "p3": 1, "p10": "Paper_airplane"}, "fire_nation"),
(4, 1, {"p1": "shivam_kapoor", "p9": 5}, "fire_nation"),
(3, 4, {"p4": "pometry"}, None),
(4, 4, {"p5": 12}, None),
(
1,
1,
{"p1": "shivam_kapoor", "p9": 5, "p10": "Paper_airplane"},
"fire_nation",
),
(2, 2, {"p1": "prop12", "p2": 2, "p10": "Paper_ship"}, "air_nomads"),
(3, 1, {"p1": "shivam_kapoor", "p9": 5}, "fire_nation"),
(3, 3, {"p2": 6, "p3": 1, "p10": "Paper_airplane"}, "fire_nation"),
(4, 1, {"p1": "shivam_kapoor", "p9": 5}, "fire_nation"),
(3, 4, {"p4": "pometry"}, None),
(4, 4, {"p5": 12}, None),
]

for time, id, props, node_type in nodes:
graph.add_node(time, str(id), props, node_type)
graph.add_node(time, str(id), props, node_type)

edge_data = [
(1, "1", "2", {"p1": "shivam_kapoor", "p10": "Paper_airplane"}, "fire_nation"),
Expand All @@ -42,6 +48,7 @@ def init_graph(graph):

return graph


def init_nodes_graph(graph):
nodes = [
(6, "N1", {"p1": 2}),
Expand Down Expand Up @@ -137,6 +144,7 @@ def init_nodes_graph(graph):

return graph


# For this graph there won't be any temporal property index for property name "p1".
def init_nodes_graph1(graph):
nodes = [
Expand All @@ -157,17 +165,15 @@ def init_nodes_graph1(graph):

return graph


# For this graph there won't be any constant property index for property name "p1".
def init_nodes_graph2(graph):
nodes = [
(1, "N1", {"p1": 1}),

(2, "N2", {"p1": 1}),
(3, "N2", {"p1": 2}),

(2, "N3", {"p1": 2}),
(3, "N3", {"p1": 1}),

(2, "N4", {}),
]

Expand Down Expand Up @@ -259,13 +265,13 @@ def init_edges_graph1(graph):
# For this graph there won't be any constant property index for property name "p1".
def init_edges_graph2(graph):
edges = [
(2, "N1", "N2", {"p1": 1}),
(2, "N2", "N3", {"p1": 1}),
(2, "N2", "N3", {"p1": 2}),
(2, "N3", "N4", {"p1": 2}),
(2, "N3", "N4", {"p1": 1}),
(2, "N4", "N5", {}),
]
(2, "N1", "N2", {"p1": 1}),
(2, "N2", "N3", {"p1": 1}),
(2, "N2", "N3", {"p1": 2}),
(2, "N3", "N4", {"p1": 2}),
(2, "N3", "N4", {"p1": 1}),
(2, "N4", "N5", {}),
]

for time, src, dst, props in edges:
graph.add_edge(time, src, dst, props)
Expand Down
Loading