Feat!: Introduce Model linting

[VaggelisD]

Fixes #3753

This PR introduces a new interface for a general purpose Model linter:

• Each Rule checks for a specific lint error; If a violation is found, a RuleViolation is created with the proper context
• Rules are grouped in RuleSets; Each RuleSet acts as both a set and a mapping (rule class name -> rule type).
• The Linter applies the RuleSets on the model and outputs warnings / raises errors depending on their severity

The linter configuration along with the default values is the following:

gateways:
 ...

default_gateway: dev

model_defaults:
  ...

linter:
  enabled: False
  
  # Can be overridden with "ALL" or ["rule0", "rule1", ...., "ruleN"]
  rules: []
  warn_rules:
  exclude_rules: ["noselectstar", "ambiguousorinvalidcolumn", "invalidselectstarexpansion"]

This is to preserve the behavior of QueryRenderer today; For this matter, validate_query is deprecated and instead the corresponding checks are implemented as rules in the Linter.

On a model granularity, rules can be ignored by the new ignore_lints attribute:

MODEL (
  name sqlmesh_example.test_model,
  kind FULL,
  ignore_lints ["noselectstar"]
);

SELECT ...;

Users can implement their own lint rules in a linter/ directory by subclassing Rule; These will be automatically picked up at load time:

# linter/user.py

import typing as t

from sqlmesh.core.linter.rule import Rule, RuleViolation, RuleSet
from sqlmesh.core.model import Model

class NoMissingOwner(Rule):
    def check(self, model: Model) -> t.Optional[RuleViolation]:
        return RuleViolation(rule=self, model=model) if not model.owner else None

    @property
    def summary(self) -> str:
        return "Model owner should be set"

[VaggelisD]

PS: v0071 is also defined here, I suspect Iaroslav's PR will be merged much sooner.

[georgesittas]

Exciting stuff!

[etonlels]

Based on this type signature it seems like rules can only check for a single violation within a model, but what if there are multiple violations of the same rule in a model? What gets shown to the end user?

[VaggelisD]

Yeah, my idea was that a RuleViolation will be indeed singular per rule but if there's multiple violations we can store their context in a collection.

For instance, if we have a rule that detects a certain faulty pattern in a query, I thought it'd look somewhat odd to have N x RuleViolation(..., expr: exp.Expression) instead of a single RuleViolation(..., exprs: t.List[exp.Expression]); The former looks like it's more polluting for now and increases the complexity which we are now trying to avoid based on internal discussions.

How does that sound?

[etonlels]

Yeah that probably works too.

[etonlels]

@VaggelisD is it possible to create configurable rules, and if so, how is that done?