Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Streaming Optimizer #41

Merged
merged 93 commits into from
Mar 12, 2025
Merged

feat: Streaming Optimizer #41

merged 93 commits into from
Mar 12, 2025

Conversation

SarveshOO7
Copy link
Contributor

@SarveshOO7 SarveshOO7 commented Mar 10, 2025

After many rewrites + discussions + considerations.

Notes:

  • The engine needs a rewrite to be able to support CPS and execution forks.
  • Storage should have versioning to allow for persistence and resource management.

yliang412 and others added 30 commits February 24, 2025 13:59
Signed-off-by: Yuchen Liang <[email protected]>
Signed-off-by: Yuchen Liang <[email protected]>
bad design; will change; puts too much pressure on memo table
@codecov-commenter
Copy link

codecov-commenter commented Mar 10, 2025

Codecov Report

Attention: Patch coverage is 57.25191% with 1456 lines in your changes missing coverage. Please review.

Project coverage is 68.3%. Comparing base (285fe5f) to head (b00e41a).

Files with missing lines Patch % Lines
optd-core/src/optimizer/handlers.rs 0.0% 319 Missing ⚠️
optd-core/src/optimizer/subscriptions.rs 0.0% 186 Missing ⚠️
optd-core/src/optimizer/ingest.rs 0.0% 183 Missing ⚠️
optd-core/src/bridge/into_cir.rs 0.0% 105 Missing ⚠️
optd-core/src/bridge/from_cir.rs 0.0% 96 Missing ⚠️
optd-core/src/optimizer/mod.rs 0.0% 84 Missing ⚠️
optd-core/src/engine/eval/expr.rs 84.4% 83 Missing ⚠️
optd-core/src/engine/mod.rs 11.6% 68 Missing ⚠️
optd-core/src/optimizer/egest.rs 0.0% 55 Missing ⚠️
optd-core/src/engine/eval/match.rs 94.3% 50 Missing ⚠️
... and 10 more
Additional details and impacted files
Files with missing lines Coverage Δ
optd-core/src/engine/eval/binary.rs 80.7% <ø> (ø)
optd-core/src/engine/eval/unary.rs 80.8% <ø> (ø)
optd-core/src/optimizer/merge_repr.rs 98.6% <98.6%> (ø)
optd-core/src/cir/rules.rs 0.0% <0.0%> (ø)
optd-core/src/engine/utils/streams.rs 90.7% <90.7%> (ø)
optd-core/src/engine/utils/tests.rs 87.7% <87.7%> (ø)
optd-dsl/src/analyzer/context.rs 75.0% <75.0%> (ø)
optd-core/src/cir/expressions.rs 0.0% <0.0%> (ø)
optd-core/src/cir/plans.rs 0.0% <0.0%> (ø)
optd-core/src/engine/eval/core.rs 90.0% <90.0%> (ø)
... and 12 more
🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AlSchlo AlSchlo changed the title WIP Integrating Cascades Driver with DSL (feat) Streaming Optimizer Mar 12, 2025
Copy link
Member

@connortsui20 connortsui20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is getting super large, and since nobody else is working on this, let's just merge this now.

@yliang412 yliang412 changed the title (feat) Streaming Optimizer feat: Streaming Optimizer Mar 12, 2025
@AlSchlo AlSchlo merged commit 03e6b1d into main Mar 12, 2025
11 checks passed
@AlSchlo AlSchlo deleted the sarvesh/cascades branch March 12, 2025 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants