Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: DSL operator codegen #25

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat: DSL operator codegen #25

wants to merge 5 commits into from

Conversation

AlSchlo
Copy link
Collaborator

@AlSchlo AlSchlo commented Feb 11, 2025

Problem

The query optimizer needs to generate type-safe Rust code for various logical and scalar operators. Manually writing these operators is error-prone and makes it difficult to maintain consistency across the codebase.

Summary of changes

Added code generation support for logical and scalar operators:

  1. Core Code Generation

    • Implemented type-safe operator generation for both logical and scalar operators
    • Automatically generates enums, structs, and helper functions
    • Handles type parameters appropriately (Value, Relation, Scalar)
  2. Generated Components (among others)

    // Generates enum definitions
    pub enum LogicalOperator<Value, Relation, Scalar> {
        Filter(Filter<Value, Relation, Scalar>),
        Join(Join<Value, Relation, Scalar>),
        // ...
    }
    
    // Generates operator structs
    pub struct Filter<Value, Relation, Scalar> {
        pub child: Relation,
        pub predicate: Scalar,
    }
    
    // Generates constructor functions
    pub fn filter<Relation, Scalar>(
        child: Relation,
        predicate: Scalar,
    ) -> LogicalOperator<OptdValue, Relation, Scalar> {
        LogicalOperator::Filter(Filter::new(child, predicate))
    }
  3. Example Usage
    Input DSL:

    Logical Filter(child: Logical, predicate: Scalar)  derive { // stuff }

    Generated code (among others):

    pub struct Filter<Value, Relation, Scalar> {
        pub child: Relation,
        pub predicate: Scalar,
    }
    
    impl<Relation, Scalar> Filter<OptdValue, Relation, Scalar> {
        pub fn new(child: Relation, predicate: Scalar) -> Self {
            Self { child, predicate }
        }
    }

@codecov-commenter
Copy link

codecov-commenter commented Feb 11, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.0%. Comparing base (44d26e2) to head (11bd4b9).

Additional details and impacted files
Files with missing lines Coverage Δ
optd-dsl/src/analyzer/semantic.rs 94.7% <100.0%> (ø)
optd-dsl/src/gen/operator.rs 98.7% <100.0%> (ø)
optd-dsl/src/parser/expr.rs 86.3% <ø> (ø)
optd-dsl/src/parser/functions.rs 90.3% <ø> (ø)
optd-dsl/src/parser/mod.rs 96.9% <ø> (ø)
optd-dsl/src/parser/operators.rs 90.4% <ø> (ø)
optd-dsl/src/parser/patterns.rs 83.0% <ø> (ø)
optd-dsl/src/parser/types.rs 89.3% <ø> (ø)

@AlSchlo AlSchlo changed the title feat: Operator Codegen from DSL feat: DSL operator codegen from Feb 11, 2025
@AlSchlo AlSchlo changed the title feat: DSL operator codegen from feat: DSL operator codegen Feb 11, 2025
Copy link
Member

@skyzh skyzh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this would conflict with #26 and need to decide who merges first

@@ -31,15 +31,6 @@ Logical Join(
schema_len = left.schema_len + right.schema_len
}

Logical Sort(child: Logical, keys: [(Scalar, String)]) derive {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider support block comments in the language?

@skyzh skyzh mentioned this pull request Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants