docs(testing): add testing guide and runnable example (#1044)#1573
Open
Hore01 wants to merge 1 commit intoapache:mainfrom
Open
docs(testing): add testing guide and runnable example (#1044)#1573Hore01 wants to merge 1 commit intoapache:mainfrom
Hore01 wants to merge 1 commit intoapache:mainfrom
Conversation
Hamilton's DAG model is the backbone of clean ETL, and tests are how that backbone stays honest. This adds a canonical guide and a runnable example that cover the four scenarios called out in apache#1044: 1. Unit-testing plain Hamilton functions 2. Unit-testing functions that use decorators (@tag, @parameterize, @extract_columns) -- both by calling the underlying callable and by building a Driver to verify the decorator wiring 3. Integration-testing the DAG with `Builder().with_modules(...).build()`, including `inputs=` and `overrides=` for short-circuiting upstream nodes 4. Driving an in-memory module via `ad_hoc_utils.create_temporary_module` for self-contained tests (e.g. of custom materializers) The docs page at `docs/how-tos/test-hamilton-code.rst` uses `.. literalinclude::` to pull every snippet from `examples/testing/`, so the guide and the example cannot drift out of sync. Closes apache#1044 Signed-off-by: Olajumoke Akinremi <106763970+Hore01@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a User Guide page on testing Hamilton code, plus a small
runnable example folder. Between them they cover the four cases in
#1044: unit-testing plain functions, unit-testing functions that use
@tag / @parameterize / @extract_columns, integration-testing with
inputs= and overrides=, and building a Hamilton module inline inside
a test using ad_hoc_utils.create_temporary_module — the same pattern
the Hamilton test suite uses for custom materializers.
Closes #1044.
The new docs page is
docs/how-tos/test-hamilton-code.rst. I slottedit into the User Guide toctree between
run-data-quality-checksanduse-hamilton-for-lineage. Every code block on the page is a.. literalinclude::pulled from the newexamples/testing/folder,so future changes to the example will flow into the docs without a
second edit.
I tested locally on Python 3.12:
pytest examples/testing/passesall 13 tests, and
ruff check/ruff format --checkare clean.Sphinx (
-b dirhtml) builds cleanly too, and no warnings referenceany of the new files. The 41 warnings I do see locally are the
autodoc warnings about optional plugin deps (dask, polars, pyspark,
mlflow, etc.) that are already present on main; CI installs the full
docsdependency group so they resolve there.A few small notes:
files since that's what the rest of the repo uses. Happy to switch
to a shorter SPDX line if that's preferred for new files.
and pytest.