Merge branch 'main' into feat/invalidate-cleanup-snapshots

Author: mday-io

docs/concepts/audits.md

@@ -158,7 +158,38 @@ AUDIT (name price_is_not_null);
 SELECT * FROM @this_model
 WHERE price IS NULL;
 ```
+### Standalone audits
 
+Standalone audits are defined independently rather than being attached to a specific model. They specify the models they depend on using the `depends_on` property.
+
+Unlike model-level audits, standalone audits can be used to validate data across one or more models without being associated with a single model.
+
+Standalone audits run as scheduled nodes during both `sqlmesh plan` and `sqlmesh run`.
+
+```sql linenums="1"
+AUDIT (
+  name assert_item_price_is_not_null,
+  dialect spark,
+  standalone TRUE,
+  depends_on (
+    sushi.items
+  )
+);
+
+SELECT *
+FROM sushi.items
+WHERE
+  ds BETWEEN @start_ds AND @end_ds
+  AND price IS NULL;
+```
+
+In this example, the audit checks that the `price` column in `sushi.items` does not contain `NULL` values for the selected date range.
+
+Standalone audits can declare dependencies using the `depends_on` property. SQLMesh can often infer dependencies directly from the audit query, but using `depends_on` is recommended when inference isn't sufficient.
+
+!!! note
+
+    Standalone audits are non-blocking only. Because they are not associated with a single model, SQLMesh cannot determine which model should be blocked if the audit fails.
 ## Built-in audits
 SQLMesh comes with a suite of built-in generic audits that cover a broad set of common use cases. Built-in audits are blocking by default, but they all have non-blocking counterparts which you can use by appending `_non_blocking` - see [Non-blocking audits](#non-blocking-audits).
 

docs/concepts/plans.md

@@ -109,6 +109,12 @@ For context, every model has a start date. The start can be specified in [the mo
 
 Because the prod environment supports business operations, prod plans ensure every model is backfilled from its start date until the most recent completed time interval. Due to that restriction, the `plan` command's `--start` and `--end` options are not supported for regular plans against prod. The options are supported for [restatement plans](#restatement-plans) against prod to allow re-processing a subset of existing data.
 
+!!! note "Explicit execution time"
+
+    "The most recent completed time interval" is measured relative to the plan's *execution time*, which defaults to now. If you pass an explicit `--execution-time` that is later than the intervals already loaded in prod, the plan extends its end date up to that time and backfills the intervals in between.
+
+    For example, if a daily model in prod is loaded through 2025-12-25, running `sqlmesh plan --execution-time '2025-12-28'` backfills the missing 2025-12-26 and 2025-12-27 intervals.
+
 Non-prod plans are typically used for development, so their models can optionally be backfilled for any date range with the `--start` and `--end` options. Limiting the date range makes backfills faster and development more efficient, especially for incremental models using large tables.
 
 #### Model kind limitations

docs/guides/linter.md

@@ -74,6 +74,7 @@ Here are all of SQLMesh's built-in linting rules:
 | `invalidselectstarexpansion` | Correctness | The query's top-level selection may be `SELECT *`, but only if SQLMesh can expand the `SELECT *` into individual columns |
 | `noselectstar`               | Stylistic   | The query's top-level selection may not be `SELECT *`, even if SQLMesh can expand the `SELECT *` into individual columns |
 | `nomissingaudits`             | Governance  | SQLMesh did not find any `audits` in the model's configuration to test data quality.                                                 |
+| `nomissingunittest` | Governance | SQLMesh did not find any `unit tests` associated with the model to test |
 
 ### User-defined rules
 
@@ -126,6 +127,14 @@ Error: Linter detected errors in the code. Please fix them before proceeding.
 
 Use `sqlmesh lint --help` for more information.
 
+You can pass `--local` to run lint without loading state from the configured state connection:
+
+``` bash
+$ sqlmesh lint --local
+```
+
+This can make linting faster in repositories where all referenced models are loaded from local files. In multi-repository setups, or when linting only a subset of projects, `--local` may cause additional linting errors because SQLMesh will not resolve references or schemas from models that exist only in remote state.
+
 
 ## Applying linting rules
 
@@ -258,4 +267,4 @@ You may specify that a rule's violation should not error and only log a warning
     )
     ```
 
-SQLMesh will raise an error if the same rule is included in more than one of the `rules`, `warn_rules`, and `ignored_rules` keys since they should be mutually exclusive.
+SQLMesh will raise an error if the same rule is included in more than one of the `rules`, `warn_rules`, and `ignored_rules` keys since they should be mutually exclusive.

docs/integrations/engines/redshift.md

@@ -29,6 +29,7 @@ pip install "sqlmesh[redshift]"
 | `region`                | The AWS region of the Amazon Redshift cluster                                                               | string |    N     |
 | `cluster_identifier`    | The cluster identifier of the Amazon Redshift cluster                                                       | string |    N     |
 | `iam`                   | If IAM authentication is enabled. IAM must be True when authenticating using an IdP                         |  dict  |    N     |
+| `db_user`               | The database user to authenticate as. Required when using IAM authentication                                | string |    N     |
 | `is_serverless`         | If the Amazon Redshift cluster is serverless (Default: `False`)                                             |  bool  |    N     |
 | `serverless_acct_id`    | The account ID of the serverless cluster                                                                    | string |    N     |
 | `serverless_work_group` | The name of work group for serverless end point                                                             | string |    N     |

docs/reference/cli.md

@@ -656,6 +656,9 @@ Usage: sqlmesh lint [OPTIONS]
 
 Options:
   --model TEXT           A model to lint. Multiple models can be linted.  If no models are specified, every model will be linted.
+  --local                Lint using only locally loaded project files without loading state. In multi-repository setups, or when
+                         linting only a subset of projects, this may cause additional linting errors because SQLMesh will not resolve
+                         references or schemas from models that exist only in remote state.
   --help                 Show this message and exit.
 
-```
+```

docs/reference/configuration.md

@@ -114,7 +114,7 @@ Formatting settings for the `sqlmesh format` command and UI.
 | `normalize`           | Whether to normalize SQL (Default: False)                                                      | boolean |    N     |
 | `pad`                 | The number of spaces to use for padding (Default: 2)                                           |   int   |    N     |
 | `indent`              | The number of spaces to use for indentation (Default: 2)                                       |   int   |    N     |
-| `normalize_functions` | Whether to normalize function names. Supported values are: 'upper' and 'lower' (Default: None) | string  |    N     |
+| `normalize_functions` | How to normalize function name casing. `false` (default) preserves the casing of custom and audit function names as written; `"upper"` uppercases all function names; `"lower"` lowercases all function names; `true` defers to SQLGlot's generator default and uppercases all function names including custom ones; `null` (or omitting the key) is excluded during serialization and therefore takes the same `false` default path — it does **not** defer to SQLGlot's generator default. Note: SQLGlot built-in function names may still be canonicalized by the parser regardless of this setting. | string \| boolean \| null |    N     |
 | `leading_comma`       | Whether to use leading commas (Default: False)                                                 | boolean |    N     |
 | `max_text_width`      | The maximum text width in a segment before creating new lines (Default: 80)                    |   int   |    N     |
 | `append_newline`      | Whether to append a newline to the end of the file (Default: False)                            | boolean |    N     |

pyproject.toml

@@ -65,6 +65,10 @@ dev = [
     # it depends on the 'InvalidCatalogIntegrationConfigError' class that only exists as of dbt-adapters==1.16.6
     # so we exclude it to prevent failures and hope that upstream releases a new version with the correct constraint
     "dbt-snowflake!=1.10.1",
+    # fastjsonschema 2.22+ uses PEP 604 type hints and now requires Python >=3.10
+    # (upstream #211 / #213). Keep 3.9 on the 2.21 line; pulled in via
+    # dbt-bigquery → nbformat.
+    "fastjsonschema<2.22; python_version<'3.10'",
     "dbt-athena-community",
     "dbt-clickhouse",
     "dbt-databricks",

sqlmesh/cli/main.py

@@ -44,6 +44,17 @@
 LOCAL_ONLY_COMMANDS = ("format",)
 
 
+class _SQLMeshGroup(click.Group):
+    def parse_args(self, ctx: click.Context, args: t.List[str]) -> t.List[str]:
+        rest = super().parse_args(ctx, args)
+        # Preserve the subcommand arguments because Click consumes them before invoking the group callback.
+        protected_args = getattr(ctx, "_protected_args", None)
+        if protected_args is None:
+            protected_args = ctx.protected_args
+        ctx.meta["subcommand_args"] = tuple(protected_args) + tuple(ctx.args)
+        return rest
+
+
 def _sqlmesh_version() -> str:
     try:
         from sqlmesh import __version__
@@ -53,7 +64,7 @@ def _sqlmesh_version() -> str:
         return "0.0.0"
 
 
-@click.group(no_args_is_help=True)
+@click.group(cls=_SQLMeshGroup, no_args_is_help=True)
 @click.version_option(version=_sqlmesh_version(), message="%(version)s")
 @opt.paths
 @opt.config
@@ -118,6 +129,9 @@ def cli(
     load = True
     # Local-only gating must hold for any number of --paths, so it stays outside the block below.
     load_state = ctx.invoked_subcommand not in LOCAL_ONLY_COMMANDS
+    # The parent callback constructs Context before Click invokes `lint`, so inspect its parsed args here.
+    if ctx.invoked_subcommand == "lint" and "--local" in ctx.meta["subcommand_args"]:
+        load_state = False
 
     if len(paths) == 1:
         path = os.path.abspath(paths[0])
@@ -539,6 +553,7 @@ def diff(ctx: click.Context, environment: t.Optional[str] = None) -> None:
 )
 @click.option(
     "--min-intervals",
+    type=int,
     default=None,
     help="For every model, ensure at least this many intervals are covered by a missing intervals check regardless of the plan start date",
 )
@@ -1199,6 +1214,12 @@ def environments(obj: Context) -> None:
     multiple=True,
     help="A model to lint. Multiple models can be linted. If no models are specified, every model will be linted.",
 )
+@click.option(
+    "--local",
+    is_flag=True,
+    expose_value=False,
+    help="Lint using only locally loaded project files without loading state.",
+)
 @click.pass_obj
 @error_handler
 @cli_analytics

sqlmesh/core/config/connection.py

@@ -1335,6 +1335,7 @@ class RedshiftConnectionConfig(ConnectionConfig):
         region: The AWS region where the Amazon Redshift cluster is located.
         cluster_identifier: The cluster identifier of the Amazon Redshift cluster.
         iam: If IAM authentication is enabled. Default value is False. IAM must be True when authenticating using an IdP.
+        db_user: The database user to authenticate as. Required when using IAM authentication.
         is_serverless: Redshift end-point is serverless or provisional. Default value false.
         serverless_acct_id: The account ID of the serverless. Default value None
         serverless_work_group: The name of work group for serverless end point. Default value None.
@@ -1360,6 +1361,7 @@ class RedshiftConnectionConfig(ConnectionConfig):
     region: t.Optional[str] = None
     cluster_identifier: t.Optional[str] = None
     iam: t.Optional[bool] = None
+    db_user: t.Optional[str] = None
     is_serverless: t.Optional[bool] = None
     serverless_acct_id: t.Optional[str] = None
     serverless_work_group: t.Optional[str] = None
@@ -1397,6 +1399,7 @@ def _connection_kwargs_keys(self) -> t.Set[str]:
             "region",
             "cluster_identifier",
             "iam",
+            "db_user",
             "is_serverless",
             "serverless_acct_id",
             "serverless_work_group",

sqlmesh/core/config/format.py

@@ -12,7 +12,21 @@ class FormatConfig(BaseConfig):
         normalize: Whether to normalize the SQL code or not.
         pad: The number of spaces to use for padding.
         indent: The number of spaces to use for indentation.
-        normalize_functions: Whether or not to normalize all function names. Possible values are: 'upper', 'lower'
+        normalize_functions: How to normalize function name casing.
+
+            * ``False`` (default) — preserves the original spelling of custom and audit
+              function names.  SQLGlot built-in functions (e.g. ``COUNT``, ``SUM``) may
+              still be uppercased because the parser discards the original token.
+            * ``"upper"`` — uppercases all function names, including custom audit
+              references.
+            * ``"lower"`` — lowercases all function names, including built-in ones.
+            * ``True`` — defers to SQLGlot's generator default, which uppercases all
+              function names including custom ones.
+            * ``None`` — excluded from the serialized generator options by Pydantic's
+              ``exclude_none`` behaviour, so ``format_model_expressions`` falls back to
+              its own ``False`` default.  Setting this in YAML as ``null`` or omitting
+              the key is therefore equivalent to ``false``; it does **not** defer to
+              SQLGlot's generator default the way ``True`` does.
         leading_comma: Whether to use leading commas or not.
         max_text_width: The maximum text width in a segment before creating new lines.
         append_newline: Whether to append a newline to the end of the file or not.
@@ -22,7 +36,7 @@ class FormatConfig(BaseConfig):
     normalize: bool = False
     pad: int = 2
     indent: int = 2
-    normalize_functions: t.Optional[str] = None
+    normalize_functions: t.Union[str, bool, None] = False
     leading_comma: bool = False
     max_text_width: int = 80
     append_newline: bool = False