Skip to content

Conversation

hatyo
Copy link
Contributor

@hatyo hatyo commented Oct 15, 2025

This introduces support for SQL views. Views are virtual tables defined by SQL queries that are compiled lazily when referenced, providing a way to encapsulate complex queries and present simplified interfaces to users.

Implementation Overview

Views are stored as raw SQL text in the catalog and compiled on-demand during query execution. The implementation follows a layered architecture:

At the core layer, we introduce RawView which represents the persisted view definition. Views are stored in the metadata alongside user-defined functions and are serialized/deserialized using the existing protobuf infrastructure. The RecordMetaData and RecordMetaDataBuilder classes now maintain a map of views that get persisted with the schema.

At the relational API layer, RecordLayerView wraps the raw view definition with compilation capabilities. Each view contains a lazy compilation function that transforms the SQL text into a logical plan when the view is first referenced. This compiled plan can be cached to avoid recompilation on subsequent accesses.

The query parser has been extended to recognize view references in FROM clauses. When a view is encountered, the semantic analyzer retrieves the view definition from the catalog and triggers compilation by invoking the view's compilation function. The resulting logical operator is then seamlessly integrated into the query plan, allowing views to be used just like regular tables.

Features

  • View Definition Storage: Views are defined using CREATE VIEW syntax and stored as SQL text in the schema template
  • Lazy Compilation: View definitions are compiled into logical plans only when referenced in queries
  • Nested Views: Views can reference other views, enabling layered abstractions
  • Complex Queries: Views support CTEs, joins, aggregations, and other advanced SQL features
  • Schema Integration: Views are first-class metadata objects alongside tables and functions

The implementation includes comprehensive testing through YAML integration tests that validate various view scenarios including simple filters, nested views, CTEs within views, and views referencing user-defined functions.

This fixes #3670.

@hatyo hatyo added relational issues related to relational FDB enhancement New feature or request labels Oct 15, 2025
@hatyo hatyo changed the title Support SQL views. Support SQL views Oct 16, 2025
@hatyo hatyo marked this pull request as ready for review October 16, 2025 14:47
@normen662 normen662 self-requested a review October 18, 2025 19:36
}
for (RecordMetaDataProto.PView viewProto: metaDataProto.getViewsList()) {
final RawView rawView = (RawView)PlanSerialization.dispatchFromProtoContainer(
new PlanSerializationContext(DefaultPlanSerializationRegistry.INSTANCE,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like piggy-backing on plan serialization to just be able to do the dispatch on a multitude of views here . How many kinds of views do we have right now (I think one) -- how many do you think we will have later?
Suggestions:

  1. extricate the logic from the plan serialization logic and make the specific plan serialization logic use and share that logic with this code.
  2. dynamic dispatch in the same way as that logic but with a switch statement if the number of actual implementations is like 1 or 2 ever.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I originally wanted to introduce a forward-looking representation of metadata that leaves the door open for introducing other flavors of views, such as fully expanded, partially expanded, and so on. But I don't think this is realistic, moreover, views in their current shape at least, are pure metadata artifacts, and you're right for calling out their useless dependence of the plan serialization.

Having said that, I changed the metadata representation of the views such that it is now a pure metadata artifacts whose Ser/De has nothing to do anymore with the plan serialization API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request relational issues related to relational FDB

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support SQL views

2 participants