Separate Operator Execution from Operator Type for Lightweight Optimization

[mdr223]

We ran into a bug with RagConvert where it created an OpenAI() client in every call to its __init__() method. This call blocked for a small amount of time, but in the "seven filters" optimization test it created a 17s runtime instead of a ~1s runtime.

In practice, the optimizer should not need to create thousands of copies of classes (which may have expensive setup logic), when all it really needs to know is the type and params of the physical operator.

This issue would come up with a good way of separating this info out from the physical operator class, to allow for more lightweight optimization.

[zacharias1219]

One good approach to realizing this separation is by creating an OperatorMetadata class that retains only the operator type and parameters, while avoiding unnecessary instantiations in optimization. These metadata objects can be used by the optimizer in place of complete operator instances, without the cost of expensive setup procedures such as API client creation in RagConvert. The principal alterations would include changing the optimizer to register and fetch operator metadata, revising PhysicalOperator to delay initialization until execution time, and modifying QueryProcessor to create operators only upon demand. This solution maintains optimization lightweight while keeping behavior at execution time unaffected.