Repository navigation
fix(api): leave post-stratification out of the API document until it is available (#963) - #2081
Open
amarkanday wants to merge 2 commits into
Open
amarkanday wants to merge 2 commits into
amarkanday wants to merge 2 commits into
Conversation
…is available (#963) POST /api/v1/results/{experiment_id}/post-stratification answers 501 for every existing experiment, so it is published as an operation that never succeeds. It is now left out of the API document: the decorator carries include_in_schema=False, and the route is otherwise unchanged. It still answers 501 for an existing experiment, 404 for an unknown one, 422 for a malformed body, and 401 or 403 as before. The two snapshots and the dashboard fixture lose the path, and the two components only it used (PostStratificationRequest and PostStratResultResponse). Nothing else in them changes. Around that: - the regression test that pinned the route as deprecated now pins it as absent from both snapshots and from the live document, with fdr-correction on the same router as the positive control; - the request's two documented defaults (metric_col, alpha) are held by a unit test on the schema instead of by the snapshot; - the fuzz exclusion for the route is deleted, as the loader refuses an exclusion for an operation that is not in the document, and test_deprecated_operations_stay_fuzzed now covers every deprecated operation in the full snapshot; - the route is taken off the docs floor, and stability.md and post-stratification.md say it is not in the API document. The route stays beta; no BREAKING CHANGE. Part of #963. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011kqtSDC7sd5ue3m6LuwdXV Signed-off-by: amarkanday <a.markanday@gmail.com>
Merges main at 4c45fcd. No conflicts; `make openapi` regenerated the snapshots and they differ from main only by this branch's change. Gates run on the merged tree: the branch's own test files, backend/tests/smoke, backend/tests/unit, scripts/doc_claims.py check (0 problems), ruff check and ruff format --check over backend/ modules/ scripts/. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NpxPyav9uBxNuiXsDvD8qn Signed-off-by: amarkanday <a.markanday@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #963.
What and why
POST /api/v1/results/{experiment_id}/post-stratificationanswers 501 for every existing experiment, yet it is published as an operation. It is now left out of the API document withinclude_in_schema=False, until it is computed from an experiment's own data. The route is otherwise unchanged: 501 for an existing experiment, 404 for an unknown one, 422 for a malformed body, 401 and 403 as before. It stays beta, so there is no!and no BREAKING CHANGE.Changed:
backend/app/api/v1/endpoints/post_stratification.py: the keyword argument, one comment line and the module docstring line. Nothing else;git diff origin/main -- backend/app/services backend/app/schemasis empty.make openapi: both snapshots and the dashboard fixture lose the path (below).test_the_post_stratification_route_is_not_in_the_published_documents(regression) replaces the test that pinned the route as deprecated. It checks both snapshots and the live core document: the path and both components are absent, andPOST /api/v1/results/{experiment_id}/fdr-correctionis present as the positive control.backend/tests/unit/schemas/test_post_stratification_schema.py: the request's documented defaults (metric_colis"metric_value",alphais 0.05) and bounds (alpha0.0 and 1.0 are refused), which the snapshot used to hold. The two claim units cite it.POST_STRATleavesEXCLUDED(the constant stays);test_deprecated_operations_stay_fuzzedis now generic (every deprecated operation in the full snapshot is selected in the superuser and viewer passes; it asserts the set is not empty); comments and docstrings reworded;unreached.tomlis untouched.tests/acceptance/docs/floor.toml: the route is taken off the floor.docs/api/stability.mdanddocs/statistics/post-stratification.mdsay the route is not in the API document, with their claim units.Snapshot diff (structural, against origin/main)
The regression test, three results
AssertionError: openapi-v1.stable.json/assert '/api/v1/results/{experiment_id}/post-stratification' not in {...}.include_in_schema=Falsedropped (snapshots left as committed): fails,AssertionError: the live core document.1 passed.The loader's refusal with the exclusion entry left in
On this tree, before the entry was deleted:
Verification
default="metric_value"->default="metric":AssertionError: assert 'metric' == 'metric_value'(1 failed, 3 passed); restored, 4 passed.AssertionError: no deprecated operation to protect; planted an exclusion ofPOST /api/v1/wizard/validate(viewer):assert [('POST /api/...e', 'viewer')] == []. Restored.pytest backend/tests/unit/scripts/test_fuzz_check.py backend/tests/unit/docs/test_docs_floor.py: 176 passed.python3 -I scripts/doc_claims.py check: exit 0, problems: 0.test_docs_only_gate.py: 47 passed.ruff checkandruff format --checkover backend/ modules/ scripts/: exit 0.backend/tests/smoke(full tree): all pass except the knowntest_version_sources(15, this venv cannot build the sdist) and onetest_wiringerror (no database). Same set in agit archive HEADcopy withmodules/removed and no.git: 1045 passed.backend/tests/unit: 25142 passed, 0 failed; the 226 errors needed a database and all pass against a scratch PostgreSQL 14 (the files that errored: 1299 and 1139 passed).backend/tests/integration/api/test_post_strat_api.pyagainst a scratch PostgreSQL 14: 20 passed (the 501, 404 and 422 answers).git diff origin/main -- backend/app/services backend/app/schemasis empty.NOT VERIFIED
unreached.toml), the docs journeys, core-build andmkdocsin CI.🤖 Generated with Claude Code
https://claude.ai/code/session_011kqtSDC7sd5ue3m6LuwdXV