Skip to content

unified: Add facade AST - #22258

Merged
asgerf merged 21 commits into
github:mainfrom
asgerf:unified/facade-ast
Jul 31, 2026
Merged

unified: Add facade AST#22258
asgerf merged 21 commits into
github:mainfrom
asgerf:unified/facade-ast

Conversation

@asgerf

@asgerf asgerf commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

This PR adds support for a "Facade AST" layer in the tree-sitter-extractor's QL generator, and a few other related tasks.

  • Add facade AST support, as a way to augment generated classes with more members.
  • Adds getA/getAn-getters for plural fields, so we can write getAnArgument() instead of getArgument(_).
  • Adds a basic facade AST for unified to exercise the new feature a bit.
  • Drive-by bugfix in local scoping which happened to come along with the AST refactoring.

Augmenting AST node classes

To augment an AST node, define a same-named class in the facade AST module, subclassing and shadowing the original class.

For example:

module FacadeAst {
  private import Ast as G
  import G

  // shadows the original Pattern class
  class Pattern extends G::Pattern {
    Pattern getEnclosingPattern() { ... } // extra hand-written member
  }
}

The facade module publicly imports the generated AST module, so any classes that have not been augmented will be exposed as-is.

Changes to the generated AST

In the generated AST, references to AST node classes now target the facade (which may resolve back to the generated AST itself, for classes that have not been augmented):

module Ast {
  private import FacadeAst as F

  class VariableDeclaration extends F::AstNode {
    F::Pattern getPattern() { ... } // Return type if F::Pattern so getEnclosingPattern is visible
    F::Expr getValue() { ... } // Resolve to the same as Expr since no facade for Expr exists yet
  }
  ...
}

Final aliases

This trick mentioned above was initially obstructed by the final aliases, making it impossible to augment a base class of another class. For example, the generated AST for AstNode was actually

private class AstNodeImpl { ... }
final class AstNode = AstNodeImpl

final class CallExpr extends AstNodeImpl { ... }

If we were to augment the AstNode class in the facade AST, the subclasses would not actually inherit its members, because they extend AstNodeImpl, which was private and could thus not be augmented.

To fix this, the layer of final aliases needs to happen after the facade AST, not in-between. There is now a separate module with all the final aliases:

module Ast {
  private import FacadeAst as F;

  class AstNode { ... }
  class Expr extends F::AstNode { ... }
  ...
}

module AstFinal {
  private import FacadeAst as F;

  final class AstNode = F::AstNode;
  final class Expr = F::Expr;
  ...
}

In unified we expose the AstFinal through unified.qll. For Ruby and QL4QL it doesn't really matter which one is exposed since the AST is hidden from end-users anyway.

asgerf added 14 commits July 30, 2026 14:52
Users are not supposed to 'import' these files directly, so putting them into 'internal'.
The split between private, non-final Impl classes and public final classes ultimately prevented the facade AST from instrumenting base classes like 'Expr' and have its subclasses actually inherit those members.

This commit takes a step towards fixing that by moving the final aliases into a separate module and making the other classes public and stripping their Impl suffix. It is up to each language to avoid leaking the non-final classes (which Ruby and QL4QL don't do anyway).
Previously each class just used 'AstNode' as its base class.

Now it mentions each of the supertypes it is part of.
Now that final classes are factored out, base classes can refer to the facade now.
Also covers a few other missed cases.
ConditionalPattern was not added to getEnclosingOrPattern()
Update the test to detect the bug

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a façade layer for extending generated unified AST classes while preserving final public aliases.

Changes:

  • Extends the AST generator with façade-aware imports, inheritance, and final aliases.
  • Adds hand-written unified AST helpers for modifiers and enclosing patterns.
  • Updates variable-binding tests and generator consumers.
Show a summary per file
File Description
unified/scripts/create-extractor-pack.sh Generates the internal AST library.
unified/ql/test/library-tests/variables/variables.ql Adds ambiguity validation.
unified/ql/test/library-tests/variables/variables.expected Records expected empty failure sets.
unified/ql/test/library-tests/variables/test.swift Tests multi-pattern bindings.
unified/ql/test/library-tests/BasicTest/test.ql Tests the public import.
unified/ql/lib/unified.qll Exposes the final façade AST.
unified/ql/lib/codeql/unified/internal/Variables.qll Uses generalized pattern traversal.
unified/ql/lib/codeql/unified/internal/FacadeAst.qll Defines hand-written façade members.
unified/ql/lib/codeql/unified/internal/Ast.qll Updates generated unified AST output.
unified/extractor/src/generator.rs Enables façade generation.
shared/tree-sitter-extractor/src/node_types.rs Makes type names cloneable.
shared/tree-sitter-extractor/src/generator/ql.rs Adds private imports and façade types.
shared/tree-sitter-extractor/src/generator/ql_gen.rs Generates façade-aware class hierarchies.
shared/tree-sitter-extractor/src/generator/mod.rs Produces façade and final modules.
ruby/extractor/src/generator.rs Selects non-façade generation.
ql/ql/src/codeql_ql/ast/internal/TreeSitter.qll Updates generated QL AST output.
ql/extractor/src/generator.rs Selects non-façade generation.

Review details

  • Files reviewed: 16/18 changed files
  • Comments generated: 0
  • Review effort level: Medium

@github-advanced-security github-advanced-security AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

@asgerf
asgerf marked this pull request as ready for review July 31, 2026 08:22
@asgerf
asgerf requested review from a team as code owners July 31, 2026 08:22
@asgerf
asgerf requested review from a team as code owners July 31, 2026 08:22
@asgerf asgerf added the no-change-note-required This PR does not need a change note label Jul 31, 2026
tausbn
tausbn previously approved these changes Jul 31, 2026

@tausbn tausbn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment, otherwise this looks good to me. 👍

Comment thread shared/yeast/src/dump.rs Outdated
Comment on lines +5 to +10
type TypeCheckContext<'a> = (
&'a Schema,
Option<&'a [crate::schema::NodeType]>,
Option<(&'a str, &'a str)>,
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this ought to be upgraded to a named struct? It's not immediately obvious to me what the various parts of this tuple represent.

@asgerf asgerf Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. Fixed in the next commit.

@asgerf
asgerf merged commit 684a32b into github:main Jul 31, 2026
131 of 132 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-change-note-required This PR does not need a change note QL-for-QL Ruby

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants