Skip to content

Graphql refactor + rolling/expanding #2090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 45 commits into
base: master
Choose a base branch
from
Open

Graphql refactor + rolling/expanding #2090

wants to merge 45 commits into from
+5,063 −1,198

Conversation

miratepuffin
Copy link
Collaborator

@miratepuffin miratepuffin commented May 9, 2025 •
edited
Loading

What changes were proposed in this pull request?

Removed functions/API changes

  • Removed _id functions from graphql as they are no longer needed
  • Removed GqlGraphs as we now have Namespace support
  • Added Namespaces and MetaGraphs so that large collections of graphs/namespaces can be paged/listed/counted

New functions

  • Added rolling and expanding to Graph, Node, Nodes, PathFromNode, Edge and Edges

Refactor

  • Renamed all graphql structs that started with GQL to make the user facing schema a bit cleaner
  • Started running queries in blocking threads to improve multi query execution and ui interactivity for large graphs

Bug fixes

  • Fixed a problem with namespaces returning loads of null paths and not returning root
  • Fixed issue with recursive writing of indexes causing the server to crash
  • changed caching policy to never kick out graphs after some timeout by default
  • Changed WindowSet to not allow zero size step
  • Fixed issue in rolling where if the step was bigger than the window size the final window would be empty.

Testing

  • Added namespaces tests to check for correct output
  • Added better testing for namespaces and metagraphs
  • Need to add tests for GQL rollings/expanding

Why are the changes needed?

Improvements in the UI timeline and also useful for Graphql users.
Multiple queries being blocked as queries were being run on the main thread.

Are there any further changes required?

The spawn blocking is probably more of a patch than a final solution, but we need to get some benchmarks going first to know what works best.

Pometry-Team and others added 30 commits May 8, 2025 14:27
@Pometry-Team
initial graph and node version
5e2c20a
@Pometry-Team
Added rolling to everything else
e9829e0
@Pometry-Team
swapped graphs to be WindowSet
4096926
@Pometry-Team
Add busy loop
3bf6d06
@Pometry-Team
remove _id functions as strings now work for both
2869251
@Pometry-Team
Renamed graphql objects to remove Gql
03cde18
@Pometry-Team
remove spawn blocking
62ea023
@Pometry-Team
tokio spawn edge
266463a
@Pometry-Team
par edges
9290f48
@Pometry-Team
Merge branch 'master' into feature/rolling_graphql
6f7de11 
# Conflicts:
#	raphtory-graphql/src/model/graph/graph.rs
#	raphtory-graphql/src/model/graph/nodes.rs
#	raphtory/src/core/utils/errors.rs
@Pometry-Team
fix graph bug and tidy tests
194bcb5
@Pometry-Team
par graph
0433e4c
@Pometry-Team
remove GqlGraphs
3b5ced5
@Pometry-Team
par metagraph
8900d5f
@Pometry-Team
partial mutable graph, to be discussed
96508f4
@Pometry-Team
par namespace
fae3fa9
@Pometry-Team
Par node
d379540
@Pometry-Team
par nodes
0889d4a
@Pometry-Team
par path
1462120
@Pometry-Team
par window set
5126fb7
@Pometry-Team
Par props
5552f50
@Pometry-Team
Fixed tests and namespaces
75a5501
@Pometry-Team
fixed test
93529c3
@Pometry-Team
mutable components
bb601bb
@Pometry-Team
Fix other test
6fc8eee
@miratepuffin
Merge branch 'master' into feature/rolling_graphql
679768a
@Pometry-Team
Swapped to enums
f07bbd8
@Pometry-Team
Fix recursive write of indexes
f194293
@Pometry-Team
fmt
8f981b3
@Pometry-Team
fixed step size o
be9e990
Pometry-Team added 10 commits May 22, 2025 18:50
@Pometry-Team
added namespaces and metagraphs
fcd9200
@Pometry-Team
fixed namespace testing
fb89d65
@Pometry-Team
turn off test_starts
3a0dd14
@Pometry-Team
updated graphql
e134f8f
@Pometry-Team
Merge branch 'master' into feature/rolling_graphql
b8de33a 
# Conflicts:
#	python/tests/test_base_install/test_filters/semantics/test_edge_property_filter_semantics.py
#	python/tests/test_base_install/test_filters/semantics/test_node_property_filter_semantics.py
#	python/tests/test_base_install/test_filters/test_edge_composite_filter.py
#	python/tests/test_base_install/test_filters/test_edge_filter.py
#	python/tests/test_base_install/test_filters/test_edge_property_filter.py
#	python/tests/test_base_install/test_filters/test_node_composite_filter.py
#	python/tests/test_base_install/test_filters/test_node_filter.py
#	python/tests/test_base_install/test_filters/test_node_property_filter.py
@Pometry-Team
fmt
d583daa
@Pometry-Team
fixed tests
e815ff7
@Pometry-Team
Found bug in rolling where we would get an empty window if the step w…
46ffde0 
…as bigger than the window
@Pometry-Team
adding tests for graph page and count
1e39482
@Pometry-Team
fmt
3b82d46
@miratepuffin miratepuffin changed the title initial graph and node version Graphql refactor + rolling/expanding May 27, 2025
ljeub-pometry
ljeub-pometry requested changes May 27, 2025
View reviewed changes
Copy link
Collaborator

@ljeub-pometry ljeub-pometry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some things to tidy up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ljeub-pometry ljeub-pometry Awaiting requested review from ljeub-pometry

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@miratepuffin @ljeub-pometry @Pometry-Team