Skip to content

Conversation

weijinglin
Copy link
Collaborator

This PR refactors the Scheduler class to introduce a more flexible and extensible workflow scheduling mechanism. The main changes include:

  • Introduced a pipeline pool using a dictionary to manage different workflow types (e.g., build_vector_index, graph_extract), each with its own GPipelineManager, flow, prepare, and post-processing functions.
  • Added a schedule_flow method to dynamically select and execute workflows based on the flow name, supporting pipeline reuse and resource management.
  • Refactored the build_vector_index and graph_extract flows to separate preparation, execution, and post-processing logic, improving modularity and maintainability.
  • Updated related utility functions (graph_index_utils.py, vector_index_utils.py) to use the new schedule_flow interface.
  • Improved error handling and logging for schema parsing and pipeline execution.

These changes lay the foundation for supporting more complex and agentic workflows in the future, while also improving the efficiency and scalability of the current pipeline execution framework.

@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Sep 13, 2025
@github-actions github-actions bot added the llm label Sep 13, 2025
@dosubot dosubot bot added the enhancement New feature or request label Sep 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request llm size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant