Skip to content

test(showcase): video 11, find the flags you can remove (#1889) - #2069

Merged
amarkanday merged 3 commits into
mainfrom
test/1889-video-11-stale-flags
Nov 20, 2026
Merged

amarkanday merged 3 commits into
mainfrom
test/1889-video-11-stale-flags

Conversation

@amarkanday

Copy link
Copy Markdown
Contributor

Video 11, find the flags you can remove (#1889). Stage 1: the storyboard, its narration, its CLAIMS rows, the setup that makes and dates its flags (S1 of the V11-V13 plan) and gate 10's stale end state (S2). No take has run; the say lines are frozen as the narration-first review (round 1) left them.

What the video shows

One dashboard scene, the flag list, on the demo seed. The departures, the sign-in and the setup, are both off camera and said in the first caption: "Signed in off camera. Four flags made off camera, dated weeks back." The demo seed and its counts are unchanged.

  1. The list: Payments kill switch (on, tagged permanent, no pill), then the three stale flags, each with its Stale pill and one reason line, then the seed's two flags (new, no pill).
  2. Legacy signup form: "On for everyone, unchanged for 30 days".
  3. Holiday banner: "Off for everyone, unchanged for 30 days".
  4. Old search ranking (on at 30%, never asked for): "Not evaluated in 30 days".
  5. Stale pressed (its exit).
  6. Payoff: only the three rows, and the hint "Stale is a suggestion: nothing is changed or archived for you. A flag tagged permanent is never stale; tags are set through the API." held still.

The browser's time_zone and locale are pinned (America/Los_Angeles, en-US): the Updated column is toLocaleDateString (PE 6).

Cut, as the plan rules:

  • The kill switch's own beat (EMV11-6, PE 5). The opening caption that says a flag was tagged permanent through the API is 19 words and is refused on length; a test holds that refusal, so the cut is forced, not chosen. The flag is still made and listed, and gate 10 holds it not stale; no beat names or focuses on it. permanent and tags are said only in the payoff beat, with the hint that draws them in frame.
  • The headline card "Remove it from code, then archive it" (check fix 6). The dashboard offers Archive now (Dashboard screens for API-only capabilities, after launch #442, feat(dashboard): archive and unarchive a feature flag (#442) #1911; frontend/src/tests/pages/feature-flag-key-and-archive.test.tsx, which a test here checks is still there), but the card stays cut: this video does not show it. archived is said only with the hint, which draws it, on screen.

The say lines (15)

  1. The flag list marks flags that look ready to remove.
  2. A flag must be at least 30 days old to be stale.
  3. The demo's own flags are new, so they are not stale.
  4. Each stale flag says why, under its Stale pill.
  5. Legacy signup form is on for everyone, unchanged for 30 days.
  6. Everyone gets one answer, so it's usually ready to come out.
  7. Holiday banner is off for everyone, unchanged for 30 days.
  8. The count starts at its last change, its Updated date.
  9. Old search ranking: not evaluated in 30 days, while others were.
  10. That means no evaluation was recorded, not that no code uses it.
  11. Press Stale to list only the flags marked stale.
  12. Stale is a suggestion: nothing is changed or archived for you.
  13. A flag tagged permanent is never stale.
  14. Tags are set through the API.
  15. Listing the stale flags changes none of them.

Sentences 5 and 7 say the page's reason lines word for word, and 9 says the page's line plus the guide's words ("while evaluations of other flags were") (staleReasonText, N the API's stale_days); 12-14 are the hint, split at its semicolon because a sentence is at most 12 words. Projection with video 9's fitted clip model (no Polly call), as the capture projects it (voiced.projected_ms, the title and end cards included): 71.9 s central, 76.6 s p90, of 90 s (refused above 87.3 s). The silent estimate is 33.1 s with the cards, under the 55 s floor: the silent take exits 1 with the length line alone (S5, EMV11-8), and nothing is padded.

Changed at the narration review, round 1, in its words: beat 3's caption is "An off flag left unchanged is stale too." (was "An off flag nobody changed is stale too.": the setup changes Holiday banner 50 days back, and the beat's own sentence says "its last change"), and sentence 15 is "Listing the stale flags changes none of them." (was "Listing them changes no flag.", where "them" read as the tags). The length test now projects with the cards, as the floor and the cap are measured.

S1: setup.flags

At load: at most six flags, each {name, key, status, rollout, tags, evaluate, created_days_ago, updated_days_ago} and nothing else (no rules, no schedule); a key the API takes ([a-z0-9][a-z0-9_-]{0,63}) and not one of the seed's (new_dashboard_ui, beta_features, held to the seed's source); updated no earlier than created; every age more than 15 days from N = 30; at least one flag evaluated; no flag named stale. The end state is derived at load from those fields by the docs' table (derived_stale), and that derivation is held equal to the server's own stale_flags.classify over every combination a setup flag can declare.

At run time (run.make_flags), after gate 14, every call with its exact status:

  1. each flag made (POST /api/v1/feature-flags/, 201) in its declared state: the create takes is_active, rollout_percentage and tags, so no PUT or toggle is sent; an answer that is not that flag refuses;
  2. one API key made under the run's token (POST /api/v1/api-keys, 201, no scope: the evaluate route takes any valid key, deps.get_api_key). Its name is showcase-setup, which is also the evaluations' user_id; the key and that name are needles from then on, and gate 6 requires both;
  3. the three flags the board evaluates asked for once each (GET /api/v1/feature-flags/evaluate/{key}?user_id=showcase-setup, 200), through offcamera.send's new keyword-only api_key=, which sends X-API-Key instead of Authorization: Bearer (check fix 5); an answer for another flag or reason refuses;
  4. all four dated back in one transaction in the stack's own Postgres (docker compose exec -T postgres psql, or the uvicorn stack's container): one UPDATE experimentation.feature_flags SET created_at = (now() AT TIME ZONE 'UTC') - make_interval(days => c), updated_at = ... (u) WHERE key = 'k' RETURNING key per flag, built only from a key matching the API's rule and two whole numbers from 0 to 3650, with -v ON_ERROR_STOP=1 --single-transaction -q -A -t; refused unless each statement returned its one row;
  5. each flag read back (GET /api/v1/feature-flags/{id}): created_at and updated_at within 10 minutes of what was set, or refused (a key matching no row, a time zone slip);
  6. before the first frame, the list and ?stale=true judged as gate 10 judges them (gates.stale_api): an evaluation the server failed to record (it swallows that failure) shows only here.

S2: gate 10's end_state.stale

After the last frame, the list and ?stale=true read again:

  • stale_days is 30 and evaluations_seen true in both (a STALE_FLAG_DAYS leak refuses, as does an answer without the fields);
  • ?stale=true is exactly the three, each with its one reason, in the API's order;
  • every other flag in the list is not stale, the seed's two and the kill switch among them;
  • each setup flag's updated_at is what the setup read back: listing changes no flag (checked before the map, so a write is named as the cause);
  • what the page drew: each setup flag's row before Stale is pressed and each stale row with it pressed shows the Stale pill and the reason lines exactly when the API says so; the hint is the page's; the pills count 3 and 3, the rows the API's total each time (6, then 3), and the notice 0.

A beat's new count keeps how many elements a CSS selector matches at its cue's start (director.count_beat); the rows and the hint are ordinary record readings. The load refuses a reading or count the end state names that no beat takes, a count named twice or as a reading, and the before/after readings taken anywhere but one beat each, in that order. Unset, stale is left out of the end state's dump, so every other video's gate 10 storyboard number is as it was (video 3's V3_DUMP pin).

The board's cut waits for the hint and for the table without the kill switch's row, so the rows read with Stale pressed are the filtered list, not the one still on screen while it reloads.

The test ids (PE 8)

test_showcase_dashboard_testids.py covers boards 9 and 11 (BOARDS, 65 and 7 ids), the count selectors included. The Stale button is found by its role and name: its test id is built from a map at run time, which the resolver reads as a number.

CLAIMS_11

One row per sentence in test_showcase_quoted_docs.py; the pages are docs/feature-flags/stale-flags.md (S), docs/capabilities.md (C) and docs/getting-started/quick-start.md (Q).

# Drawn Pages
1 flags-table; the pill's markup C: "the flag list marks a flag that looks ready to remove, and says why"
2 flags-table; N from data.stale_days S: "created at least N days ago"; "N is 30 days by default"
3 flags-table, flag-row Q: the demo seed "(three experiments, two flags)"; S: "created at least N days ago"
4 the reason list's markup and staleReasonText S: "marks such a flag as stale and says why"
5 flag-stale-reasons; caption; the on_for_everyone line S: the on_for_everyone row
6 flag-row S: "served everyone the same answer for weeks ... is usually ready to come out of your code"
7 flag-stale-reasons; caption ("left unchanged"); the off_for_everyone line S: the off_for_everyone row
8 flag-row; the Updated column and formatDate(flag.updated_at) S: "The count starts from the flag's updated_at."; "every change to a flag sets it"
9 flag-stale-reasons; caption; the not_evaluated line S: the not_evaluated row
10 flag-stale-reasons S: "Not evaluated means no evaluation was recorded, not that no code uses the flag."
11 flag-stale-filter; the Stale filter; stale: true sent only while pressed S: "?stale=true lists only the stale flags"
12 flag-stale-hint; caption; the hint S: "It is a suggestion: listing changes no flag, and nothing is archived or turned off for you."
13 flag-stale-hint; the hint S: "A flag tagged permanent is never stale."
14 flag-stale-hint; the hint S: "Tags are set through the API"
15 flags-table; the hint S and C: "Listing changes no flag"

A test also holds the guide's reasons table (its rows, their order, the permanent rule, N's default) as the one the board derives from, so a docs-only change to it meets the board.

The copy check's pathspec, generated from CLAIMS_11 and the board's cards (none): frontend/src ':!frontend/src/tests' docs/capabilities.md docs/feature-flags/stale-flags.md docs/getting-started/quick-start.md.

Plants (each red, then restored; the tree equal to the commit after each)

  • The ?stale=true check before the first frame removed: the setup's drop-one-evaluate and STALE_FLAG_DAYS plants pass, 2 FAILED.
  • The read-back's 10 minutes made unbounded: seed as shipped (no backdate) and a 7-hour skew pass, 2 FAILED. The one-row-each check removed: a key matching no row passes, FAILED.
  • api_key= sending the Bearer token too: FAILED. The SQL key check removed: a quote, capitals, a space and an empty key reach the statement, 4 FAILED.
  • The counts compare removed: the pill's test id dropped, the unfiltered rows read and the notice shown pass, 3 FAILED. The reason-line compare removed: a renamed reason passes, FAILED.
  • The updated_at check removed: a later write to a stale flag and a toggle of the kill switch pass, 2 FAILED. The stale_days check removed: an API at STALE_FLAG_DAYS=1 (which marks the same three) passes, 2 FAILED.
  • The derived-map compare removed at load: three wrong end states load, 3 FAILED. The derivation ignoring permanent: the board itself is refused ("the docs' table gives {'payments-kill-switch': ['on_for_everyone'], ...}"), 76 FAILED.
  • The dump's drop removed: test_video_3s_end_state_dump_is_mains and the every-other-board dump test FAILED.
  • count_beat not called in the beat: FAILED. The setup's needles not added to the redactor, and gate 10's stale block skipped: FAILED each.
  • On disk: data-testid="flag-stale-pill" renamed in the flag list: the testids test FAILED naming flag-stale-pill (with the CLAIMS and words tests, 12 FAILED). The hint reworded: the words test and CLAIMS_11 FAILED.
  • On the board: the zone pin removed: 4 FAILED. A sentence with no CLAIMS row: "scene flags beat 3 says 'It has been off since the holidays ended.', which has no CLAIMS row". "Nothing is archived for you." said in beat 5: the hint-words test FAILED, and the transcript grep counts 2 lines with archiv (clean: 1).
  • After the last edit (a flag renamed), four were planted again and went red as before: the read-back's bound, the reason-line compare, the updated_at check and the dump's drop (7 FAILED).
  • After the narration review: beat 3's old caption put back: 5 CLAIMS_11 tests FAILED on "scene flags beat 3 does not draw cue 'left unchanged' while it says 'Holiday banner is off for everyone, unchanged for 30 days.'". The length check, in memory: three sentences added to beats 2-4 leave the card-less p90 at 81.6 s, under the 87.3 s cap, while the projection with the cards is 88.6 s and refused.
  • Kept as permanent tests that assert the refusal: every load refusal above (28 cases), the setup refusals (5 plants, 4 other answers, a flag in another state, a stack that cannot date), the gate 10 refusals (8 screen, 4 API, a setup never read back, an empty list, a stale seed flag, reordered reasons, missing fields, a short page).

Rehearsed, not in CI

The dating step's exact statements and options, run against a scratch PostgreSQL 14 (server time zone America/Los_Angeles): psql printed the four keys, one a line, and nothing else; each flag was 120/120, 90/60, 75/50 and 60/50 days back, to the 1/10000 of a day; a key matching no row printed only the others; a failing statement in the same call rolled the transaction back (exit 1).

Other changes

  • VIDEOS registers 11 right after 5 (no pin on the order moves). When video 12's pull request is also merged, 12 takes the place right after 5 (its own pin) and 11 follows it: this board's pin asks only for a place between 5 and 6.
  • The gains pin counts seven videos at the default 4.5 dB; video 11's own gain is measured from its voice sitting and lands before the recording tag (EMV11-4). The zone pins name videos 9 and 11. overrides.yaml 11-stale-flags: {}.

Runs

  • backend/tests/unit/showcase/ plus backend/tests/unit/infrastructure/test_docs_only_gate.py: 2029 passed, 0 failed, 0 skipped (exit 0; main: 1898); scripts/check_junit_skips.py: exit 0. The same suite in a git archive HEAD copy with no .git: 2029 passed.
  • New or changed: test_showcase_stale_flags.py 112 (new), test_showcase_quoted_docs.py 24 (18 on main), test_showcase_dashboard_testids.py 11 (4).
  • Refresh onto main 373744b (2026-11-19): merge clean, the same suites 2030 passed; one board test follows main (the archive pin's new wording), the yaml comment about the archive action is corrected; with video 12's board also merged, 2117 passed.
  • ruff check and ruff format --check over the Makefile's lint scope: exit 0.
  • gitleaks detect --log-opts=origin/main..HEAD: exit 0. scripts/leak_guard.py --range origin/main..HEAD: exit 0.

Not verified

  • No take, browser, render or stack has run. The locators, the table fitting the page area at its cue's start and end (estimated about 860 of 940 px tall with six rows), the Stale button by role and name, the cut's wait for the filtered table, and innerText's shape of a row are read from the source, not seen.
  • The dating step on the Quick Start's PostgreSQL 16 and through docker compose exec: rehearsed on PostgreSQL 14's psql over TCP only.
  • The real API's answers: the setup and gate tests use a fake API whose lists are made by the server's own classifier.
  • That the demo seed records no flag evaluation of its own (it would not change the answer: three flags are evaluated).
  • The lengths: the projection uses video 9's fitted model, not real clips; voiced, with the cards, it has 16.9 s over the 55 s floor at central, and p90 is 10.7 s under the cap.
  • The frontend suite and make lint's other steps: no frontend file changes.

Closes #1889

🤖 Generated with Claude Code

https://claude.ai/code/session_01NpxPyav9uBxNuiXsDvD8qn

amarkanday and others added 3 commits November 10, 2026 20:25
Video 11's storyboard, its CLAIMS rows, the setup that makes its flags and
dates them back, and gate 10's stale end state (S1 and S2 of the V11-V13
plan, with PE 3, 5, 6 and 8). Stage 1: no take has run.

- Storyboard 11-stale-flags: the flag list on the demo seed, with four
  flags the setup makes off camera and dates back, said in the first
  caption. One beat per stale row, each saying the page's reason line;
  then Stale pressed, and the hint (the payoff), said in the page's words.
  The browser's zone and locale are pinned (the Updated column is
  toLocaleDateString). 15 sentences.
- The kill switch's own beat is cut: the opening caption that would have
  said it was tagged permanent is refused on length (EMV11-6), and a test
  holds that refusal. The archive card is cut: the dashboard has no
  archive action. Both words are said only with the hint, which draws
  them, on screen.
- setup.flags (S1): at most six, keys the API takes and the seed does not
  use, ages clear of 30 days, updated no earlier than created, and the end
  state derived at load from the flags' fields by the docs' table, held
  equal to the server's own classifier. At run time each flag is made in
  its state, one API key named showcase-setup is made (a needle, with its
  name), three flags are evaluated with it as X-API-Key (offcamera.send's
  new keyword-only api_key), all four are dated back in one transaction in
  the stack's own Postgres from checked keys and whole days only, read
  back within 10 minutes, and ?stale=true is judged before the first frame.
- end_state.stale (S2): after the last frame, stale_days 30, ?stale=true
  exactly the three with their reasons, every other flag not stale, each
  setup flag's updated_at as the setup left it, and what the page drew:
  each row's pill and reason lines, the hint, the pills 3 and 3, the rows
  the API's total, no notice. A beat's new count reads the pills, rows
  and notice by CSS. Unset, stale is left out of the end state's dump.
- test_showcase_dashboard_testids.py covers boards 9 and 11, the count
  selectors included (PE 8). CLAIMS_11 in test_showcase_quoted_docs.py.
- VIDEOS registers 11 right after 5, so no pin on the order moves; the
  gains pin counts seven videos at the default until video 11's own gain
  is measured; the zone pins name videos 9 and 11.

Closes #1889

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019CNVFcEPovQW1L3b1vWqGH
Signed-off-by: amarkanday <a.markanday@gmail.com>
The narration-first review of video 11 (round 1, approve with changes)
froze two lines as it worded them, and asked for the length check to
count the cards. Every other say line and caption is unchanged.

- Beat 3's caption is "An off flag left unchanged is stale too." (was
  "An off flag nobody changed is stale too."): the setup changes Holiday
  banner 50 days back, and the beat's own sentence says "its last
  change". Sentence 7's CLAIMS_11 row now also holds that the caption
  says "left unchanged".
- Sentence 15 is "Listing the stale flags changes none of them." (was
  "Listing them changes no flag."): said right after "Tags are set
  through the API.", "them" read as the tags. Its CLAIMS_11 row is
  re-keyed; what it holds drawn and the pages it quotes are unchanged.
- The length test projects with voiced.projected_ms, the capture's own
  projection, which counts the title and end cards: the 55 s floor and
  the cap are both on that length, so the card-less figure left the cap
  check 7 s loose. With video 9's fitted clip model: 71.9 s central,
  76.6 s p90, against the 55 s floor and the 87.3 s cap.

Refs #1889

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019CNVFcEPovQW1L3b1vWqGH
Signed-off-by: amarkanday <a.markanday@gmail.com>
Brings video 11's board (stale flags) up to main 373744b.
The merge itself was clean. One board test followed main: the stale-flags
board's archive-card check read a dashboard test that said the dashboard has
no archive action; main now offers Archive (#442, #1911), so the check reads
that pin's new wording and the card stays cut. The storyboard's comment that
said the dashboard has no archive action now says it has one and the card is
cut by decision. The narration is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NpxPyav9uBxNuiXsDvD8qn
Signed-off-by: amarkanday <a.markanday@gmail.com>
@amarkanday
amarkanday merged commit d7918c1 into main Nov 20, 2026
69 checks passed
@amarkanday
amarkanday deleted the test/1889-video-11-stale-flags branch November 20, 2026 02:34
amarkanday added a commit that referenced this pull request Nov 20, 2026
Brings video 12 (balance within groups) up to main d7918c1,
which merged video 11 (#2069). The same 5 files are touched by both videos;
each keeps both videos' entries, nothing from either side is dropped:

- tests/acceptance/showcase/capture/config.py: VIDEOS lists 05-plan-and-team,
  12-balance-within, 11-stale-flags, then 06-shoplab-ab-test.
- tests/acceptance/showcase/overrides.yaml: both entries, 11-stale-flags and
  12-balance-within, each empty.
- backend/tests/unit/showcase/test_showcase_dashboard_testids.py: BOARDS holds
  both boards (video 11: 7 ids, video 12: 23), both videos' sections stay, and
  the docstring names both.
- backend/tests/unit/showcase/test_showcase_voice.py: the gains pin is now 8,
  since both videos take the default 4.5 dB; the comment says so.
- backend/tests/unit/showcase/test_showcase_quoted_docs.py: both quoted-docs
  blocks (CLAIMS_11 and CLAIMS_12 with the card's key) and both docstring
  paragraphs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NpxPyav9uBxNuiXsDvD8qn
Signed-off-by: amarkanday <a.markanday@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Site video: find the flags you can remove

1 participant