Repository navigation
Conversation
The Bills, Clients and Lobbyists lists and the bill page each had their own SortTh. They now share components/lobbying/SortTh.tsx, along with the header style. The header's label is an unstyled button, so it can be reached with Tab and sorted with Enter or Space, and the sorted column has aria-sort. Every sortable column shows an arrow (↕ until it is sorted), so headers no longer change width when the sort changes; only the bill page did that before. Verified: npx jest components/lobbying (SortTh.test.tsx: keyboard sort, aria-sort, ↕ on unsorted columns); tsc --noEmit shows no errors in the files changed; prettier --check; next lint adds no warnings. Not verified: sorting on the four pages in a browser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…onents The redesigned Bill and Client pages group filings by organization or by bill, per half-year reporting period, and show a position bar and an expandable "who lobbied" table. This adds those pieces, tested, for the page rebuild in #90; no page uses them yet. - activity.ts: normalizePosition, now the only one, follows the scraper's rule, so the precomputed counts and the pages agree. Existing pages now show "Monitor" filings as Neutral. periodOf takes the half-year from the filing, then its registrant, then falls back to the year. groupActivity builds the rows, with in-house status per client and a firm's agents on that client; orgPositionBreakdown counts each organization once, by its latest period. - PositionBar: support/oppose/neutral in the MAPLE position colors, plus a new purple for "multiple positions", with counts on hover, in its label and in an optional legend. - ActivityTable: rows open with a button to each filer's position per period and SoS filing links; it scrolls inside its box on narrow screens. - fetchRegistrantsByIds fetches registrants in batches of 30. The lobbyingClients hooks, which read a collection nothing writes, are removed. Closes #89 Verified: npx jest components/lobbying (37 tests, incl. mixed positions across periods, a filer with no period data, an org in-house for one client and a firm for another, opening a row by keyboard); tsc --noEmit shows no errors in the files changed; prettier --check; next lint adds no warnings. Not verified: PositionBar and ActivityTable in a browser or at phone width (no page uses them until #90); fetchRegistrantsByIds against Firestore; Monitor filings showing as Neutral on a live page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The six Lobbying Explorer pages now follow the redesign canvas's layout, labels and interactions in MAPLE's colors. - Overview: key figures per session, a payments-by-session chart whose bars pick the session, and the most-lobbied bills (overall for all sessions). - Bills and Clients lists: the redesigned columns (organizations with position bars; bills, lobbyists split into in-house and firms with their agents, and reported paid). - Bill and Client pages: a summary and an expandable organization or bill table, sortable and filtered by position. - Lobbyists list: the shared table style and a count. Every session select opens on, and marks, the session in progress. To keep pages light, the Overview and the Client page are prerendered with their data. The Clients list embeds only the latest session and loads others through a CDN-cached API route; all regenerate daily. A Client page reads only the selected session's filings. Firestore's "client is offline" error reads as the server being unavailable. Closes #90 Verified: npx jest components/lobbying (59 passed), tsc --noEmit (only the 4 typesense errors from local node_modules), next lint, prettier --check. In the browser, against a full local scrape in the emulator: H4746's 105 organizations and 250 filings, the Greater Boston Chamber of Commerce's 194th summary, the 194th's 5,650 bills, sorting both ways, session and position filters, picking a session from the chart, keyboard expand and row clicks, fixed column widths through re-sorts, and phone width. That also covers the checks 5e14038, #89 and 06b1902 left open (the sortable headers, PositionBar and ActivityTable in a browser and at phone width, and the Clients page showing inHouseLobbyists and hiredFirms). Not verified: next build and static regeneration (the local build stops on the typesense type error); the pages on dev or prod, which need the #106 deploy; Firefox's session select width (checked in Chromium only); All sessions for the largest clients' pages (the emulator rejects the ~25 MB response). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Positions are now blue for support, orange for oppose and grey for neutral, as on the redesign canvas, in place of MAPLE's green, orange and navy. One table in chartTheme holds each position's bar, text, chip and square colors. Position bars, legends, the Bill page's counts, the chips in the tables and the lobbying card on MAPLE's bill pages all read from it. Text colors are all 4.5:1 or better on their backgrounds. The canvas's neutral grey is 2.1:1 as a bar segment; each segment's count is also in its hover text and the bar's label. Verified: npx jest components/lobbying, tsc --noEmit (only the 4 local typesense errors), next lint, prettier --check. In the browser on the full local scrape: the Overview's top bills, H4746's summary bar and counts, and the chips in its organization table. Not verified: the lobbying card on MAPLE's bill pages in a browser (the local bill pages 404 without bill data). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An expanded organization (Bill page) or bill (Client page) now lists every report period of its session that has ended, newest first, for each filer. Each period is a lettered square in the position's color (S, O, N), or an empty dashed square when the filer didn't file that period, as on the redesign canvas. The periods used to be only those the row's own filers filed in. A new period's lane stays empty until its reports come in. sessionPeriods and sessionPeriodsWith in activity.ts build the list, and each period carries a short label for the captions. Verified: npx jest components/lobbying (new tests for the period list and the empty squares), tsc --noEmit (only the 4 local typesense errors), next lint, prettier --check. In the browser on the full local scrape: H4746's ACLU and Abbott rows show Jan–Jun 26, Jul–Dec 25 and Jan–Jun 25 with filled and empty squares. Not verified: a Client page's All sessions view, where each bill row shows its own session's periods. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ganizations only - Banner: the Lobbying Explorer's beta banner is one centered sentence with the email inline, as on the canvas. The shared Banner is a flex row, which broke it into three lines; the lobbying banner now uses Bootstrap's theme classes on a paragraph, and the other banners are unchanged. - Bill page title: it stays the page's h1 at h3 size, since a bill's title can run long. The Client page's name keeps the full h1 size. - Bills list: only the Organizations column sorts. Verified: tsc --noEmit (only the 4 local typesense errors), next lint, prettier --check. In the browser on the full local scrape: the banner on one line at 1280px and wrapping at 390px; H4001's title at 30px (24px on a phone); the Bills list sorting by organizations both ways with a plain Bill header. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # components/db/lobbying.ts
The pages now read what compute_stats precomputes on fix/88 instead of working it out in the browser: - The Bills list and the Overview's most-lobbied bills draw each bar from every organization counted once (orgLatest*), with a "multiple" segment, so they match the Bill page. - The Bill and Client pages leave out the "No client" placeholder and H0/S0 rows, by the scraper's own rules. - Lobbyist totals from the 189th session on are exact: individuals plus hired firms that named no agents. - The Client page takes its in-house filers from its summary. - The Clients list reads one precomputed doc per session. - The Lobbyists list now loads per session like the other lists. It embeds only the current session and loads others through a new cached /api/lobbying/firms/[session] route, so its page data drops from 2.29 MB to 514 kB. Along the way, every lobbying data hook reports a result only for its current arguments, the three list API routes share one handler, and bill rows carry only the fields the lists show. Verified: npx jest components/lobbying (65 passed), tsc --noEmit (only the 4 local typesense errors), next lint, prettier --check. In the browser on the full local scrape rebuilt with fix/88: H4746 8/23/67/7 and H4001 100/5/266/13 on the Bill page, Bills list and Overview alike, with no "No client" rows; H1 first in the 194th; Abbott Laboratories "1 in-house · 1 firm (3 agents)"; the Greater Boston Chamber of Commerce's 39 bills, 6 in-house lobbyists and its own SoS page; Lobbyists 2,164 (194th), 2,009 (191st) and 8,146 (all); session switches show loading, then the new rows. That covers the check 369d878 left open (the #90 pages reading the new fields). Not verified: next build and static regeneration (the local build stops on the typesense type error); the CDN caching of the new clients and firms routes; the pages on dev or prod, which need the #106 stats run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The scraper lists the portal's "No client" again, so readers can see lobbyists who reported no client, but still leaves it out of client and organization counts. The pages now do the same: - The Bill page keeps its "No client" row, tagged "Not an organization", and leaves it out of the organization count and the position bar, so they still match the stored counts. Its filings and filers still count. - The Clients list sorts it in with the rest, tagged, with only its people count under Lobbyists, and leaves it out of "N clients". - Its own Client page says it isn't an organization in place of the registration line and SoS link, and drops the in-house clause. - Filers of "No client" show no in-house or firm label, and the Lobbyist page tags it in a lobbyist's client list. The legacy total-salary row is still dropped everywhere. Verified: npx jest components/lobbying (66 passed), tsc --noEmit (only the 4 local typesense errors), next lint, prettier --check. In the browser on the local scrape rebuilt with d5d8fab: H4576 (194th) shows 1 organization and an oppose-only bar, plus the "No client" row (Gregory P Vasil, Jackie McCarthy); H4746 still 8/23/67/7 and H1 413 organizations; the 194th Clients list still 1,810 clients, with "No client" at 48 bills, 23 lobbyists and $81,725; its Client page and Jackie McCarthy's page as described. Not verified: the pages on dev or prod, which need the #106 stats run to write the placeholder flag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As in the redesign, an expanded row's filer box now sits under the Most recent position, Who lobbied and Filings columns, not across the whole row. Each filer's period squares line up with the position chips, and its name, kind and filing links with the Who lobbied text. The position column widens from 180 to 300 px, the redesign's minimum, so a session's three period squares fit on one line. Verified: in the browser on the local scrape, on H4746 (194th) and Abbott Laboratories' Client page: squares at 453 px under chips at 452 px, filer names at 752 px under the Who lobbied text at 752 px; prettier --check. Not verified: phone width, where the table now scrolls from 840 px instead of 720 px. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in fix/88's merge of upstream main: filings record their reporting period (codeforboston#2308), the scraper deploy workflow (codeforboston#2309) and the slimmer beta banner (codeforboston#2304). The redesigned Bill and Client pages stay as they are: upstream's period sort changed the old filings tables they replaced. The Lobbyist page takes upstream's period sort, and its filings table labels rows by period. The Lobbying Explorer uses upstream's slim beta banner in place of the redesign's. Verified: tsc --noEmit (only the 4 local typesense errors), jest components/lobbying (70 passed), next lint, prettier --check. In the browser on the local scrape: the slim banner; Jackie McCarthy's Lobbyist page labeling rows "Jan–Jun 2026"; the H4746 Bill page loading its organizations; no page errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Period labels came from three places: the filings tables' formatter
("Jan–Jun 2025"), the activity rows' own month strings, and the Lobbyist
page's date ranges ("1/1/2026 – 6/30/2026"), so that page labeled the
same period two ways. All of them now use the filings tables' formatter,
which gains a short-year form ("Jan–Jun 26") for the period squares.
Verified: npx jest components/lobbying (71 passed), tsc --noEmit (only
the 4 local typesense errors), next lint. In the browser on the local
scrape: Jackie McCarthy's Lobbyist page reads "Jan–Jun 2026: $9,542" and
the like with no date ranges left; H4746's period squares read
"JAN–JUN 26" with "Jul–Dec 2025: Support" on hover; no page errors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in fix/88's code review fixes: the stats size check before writing, the settled-year rule for the weekly summary cache, and amendment chains in the migration. No frontend changes. Verified: the scraper suite (194 passed), tsc --noEmit (only the 4 local typesense errors), jest components/lobbying (71 passed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Firestore indexes every field by default and allows 40,000 index entries
per document, a limit the emulator doesn't enforce. The per-session
Lobbyists and Clients list docs held one indexed field per row, so the
dev stats run failed writing firmList_192, and firmList_193 and
clientList_all would have failed after it.
The list docs now hold their rows under one "rows" map, which
firestore.indexes.json exempts from indexing; nothing queries these
docs, each is read whole by id. The stats run's pre-write check now
estimates each document's index entries next to its size, stops before
writing anything if one passes the limit, and logs each doc's count.
The filing counts and inHouseFilers docs, which the current pages read,
keep their shape: the largest is about 10,600 entries.
Readers of these docs must read rows from "rows"; only the unmerged
redesigned list pages do.
Verified: pytest in lobbying-scraper (200 passed), including the
estimate, the stop before writing, and a check that the exempt field is
exempt in firestore.indexes.json. On a copy of the local emulator data,
the estimate puts firmList_192 (42,979), firmList_193 (40,266) and
clientList_all (55,722) over the limit and every clientList_{gc} under,
matching what dev wrote and refused. A full stats rebuild against that
copy wrote every list doc under "rows" with no errors.
Not verified: the stats run on dev once the exemption is deployed; that
Firestore applies the exemption to the rows' subfields as documented.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stats step ran out of memory on dev at Cloud Run's 512 MiB default. deploy.sh now creates and updates the job with 2 GiB and 1 vCPU, about three times the measured peak, so a recreated job or the prod job don't fall back to the default. Verified: a full local stats rebuild (592,149 filings) peaked at 688 MiB resident memory. Not verified: deploy.sh against a project; the job's peak memory on Cloud Run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stats run now writes the Lobbyists and Clients list docs as
{rows: {normalized name: row}}, so their rows stay out of Firestore's
index entry limit. fetchFirmList and fetchClientList read the rows from
there; every list page and list API route goes through them.
Verified: against the local emulator rebuilt with the merged scraper,
the firms and clients API routes return 2,164 (194), 2,241 (192), 1,811
(194) and 5,572 (all) rows, matching the stats log, and the Lobbyists
and Clients pages render with their current session's rows.
Not verified: the pages in a browser or at phone width; against dev,
which has the flat list docs until its stats run succeeds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
dasanchez
marked this pull request as ready for review
October 28, 2026 17:23
dasanchez
requested review from
Mephistic,
alexjball,
mertbagt,
mvictor55,
nesanders,
sashamaryl and
timblais
as code owners
October 28, 2026 17:23
The 2014-2018 layout (also used by recent individual disclosures that report no activity-level compensation) rebuilds each client's amount from its "Total amount paid by client" panel and dropped any client at $0. A lobbyist's $0 client then vanished from "Clients represented". The legacy 2009-2013 path had the same filter. A client with a recorded $0 now keeps its compensation entry. Fixture totals are unchanged; the 2011 and 2016 fixtures gain their $0 clients (3 and 1). Verified: scraper tests pass; the new test fails without the fix. Parsing Adam Romanow's live Jul-Dec 2025 disclosure took the Hybrid path and dropped Massachusetts Brewers Guild before the change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The overview's per-session client count came from filings, so a client named on a disclosure's compensation but with no activity rows was in the Clients list and not in the count (1,759 against 1,810 on dev for the 194th). Both now come from the client summaries, less the "No client" placeholder. Verified: scraper tests pass; the new test fails without the change. After a rebuild from the local archive into a fresh emulator, every session from 184 to 194 has stats clients equal to its list rows minus the placeholder, and clientList_all matches totalClients. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
An Executive activity has no bill number; the portal's identifier cell names the agency (for example "Office of the Governor"), but the filing doc dropped it, leaving the row with only a title. Filing docs now carry `agency` for Executive items and null for bills. Doc ids are unchanged, so existing docs are overwritten in place by a rewrite from the archive. Closes #121 Verified: scraper tests pass; the new test fails without the change. In the local archive rebuild, Adam Romanow's 5 Executive filings carry agency and no bill row does. His 2025 rows read "Office of the Governor"; his 2024 rows read "Other", the portal's own text. Not verified: dev. Its docs stay as they were until the migration's write phase and a stats run are done there. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A lobbyist's activity table showed "—" under Bills for items with no bill, such as Executive items like a chapter of the Acts, even when a position was reported on them. The cell now names the item, "Executive" (with the agency when the portal names one, e.g. "Executive · Office of the Governor") or "Other", with its title beneath. The portal's literal "Other" agency is not shown. On a lobbyist's page these rows now sort after the year's bills instead of first. Verified: yarn test:website; in yarn dev:local against an emulator rebuilt from the archive, Adam Romanow's page at 1280 and 375 px shows his 2025 rows as "Executive · Office of the Governor" with the title, his 2024 rows as "Executive" with the title, each after the year's bills. Not verified: on dev, where filings carry no agency until the migration's write phase runs; rows show "Executive" and the title until then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each row counted as a "filing" is one activity on a disclosure (a bill or an Executive item with a position), so a lobbyist with 5 disclosures showed "87 filings". Counts, column headings and empty-table text now say "activities", the portal's own name for these rows; "filing" and "disclosure" keep meaning a whole report. Only the wording changes: keys and code identifiers stay. Drops billCard.filingCount_*, which nothing used. Verified: yarn test:website; in yarn dev:local, Adam Romanow's page reads "87 activities", the Lobbyists list heads its column "Activities", and a bill page reads "…in 867 activities by 299 lobbyists and firms." Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A note under the list says what "—" and "0" mean in Clients represented and Activities, and a "—" cell carries a short tooltip. Verified: in yarn dev:local at 1280 px the note shows under the list. Not verified: the note's meaning in a session view. That view still shows "—" for a lobbyist who filed empty disclosures that session (e.g. Brian Delburn in the 194th), where the note says "0"; a per-session flag from the scraper's stats fixes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A session's Lobbyists list showed "—" for anyone with no clients and no activity that session, including a lobbyist who filed disclosures of their own that reported nothing. The list can't then tell "registered, but an entity files for them" from "filed, but reported no clients". Each row of a session's list now carries filedInSession: whether the lobbyist or entity filed a disclosure of its own that session. Verified: scraper tests pass; the new test fails without the change. After a stats rerun on the emulator rebuilt from the archive, Brian Delburn's 194th row has filedInSession true with 0 clients and 0 activities; rows without it drop from 921 to 486 in the 194th (642 to 117 in the 184th), and no row has clients or activities without it. Not verified: dev, until its stats run after deploy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A session's Lobbyists list showed "—" for anyone with no clients and no activities that session, including a lobbyist who filed disclosures of their own reporting nothing, which the note under the list says shows as 0. The list now reads the stats' filedInSession flag: filed that session, the counts show (0 when empty); registered only, "—". Rows written before the flag fall back to the old rule. In All sessions, one that filed but never reported an activity shows 0 activities rather than "—". The note now says "in the sessions shown". Closes #122 Verified: yarn test:website; in yarn dev:local against the emulator with the new stats, the 194th view shows Brian Delburn as 0 / 0 (was — / —) at 1280 and 375 px, and a registered-only lobbyist (Alexander Furey) keeps —. Not verified: dev, which keeps the old rule until its stats run writes filedInSession; the All sessions view in a browser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A client page's "reported paid" covers the session, but its SoS registration link opens one year's registration page, so 2U Inc. showed $21,250 for the 194th (2025 and 2026) and a link to a page listing $10,000 (2026 only). The link now reads "2026 SoS registration", from the client's last registered year; without one it keeps the plain label. Verified: yarn test:website; in the emulator data the linked page's year equals the last registered year for all 5,762 clients with a link; in yarn dev:local, 2U's page reads "2026 SoS registration" at 1280 and 375 px. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mephistic
approved these changes
Oct 31, 2026
Mephistic
left a comment
Mephistic
left a comment
Collaborator
There was a problem hiding this comment.
LGTM - I just want to keep an eye on our Firestore usage metrics as we promote this more
I'll hold off merging this until Monday morning - then we can co-ordinate rolling this out to PROD
The Overview's Key figures heading read "Key figures · 194th General
Court (2025–26)". The session stays in the heading and its years move to
a second, lighter line in full ("2025–2026"). For All sessions the line
gives the span of sessions with data ("2005–2026").
Verified: screenshots of the panel at 1280 and 375 px for the 194th, and
at 1280 px for All sessions, against yarn dev; yarn test:website passes;
check-types shows only the 4 known errors; no new lint warnings.
Not verified: other sessions in the select, beyond the shared helper.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Overview's payments chart named each bar by its session ("194th").
It now gives the session's years ("2025–26"); the tooltip keeps the full
"194th General Court (2025–26)". At phone width the chart labels every
third bar, which is fine as it is.
Verified: screenshots of the chart at 1280 and 375 px against yarn dev;
check-types shows only the 4 known errors; prettier passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each segment of the position bar carried its count in a title attribute, which the browser shows only after about a second. The segments now use react-bootstrap's tooltip, which shows at once (its default delay is 0), above the segment. Screen readers still read every count from the bar's label. This covers the bar on the Overview's top bills, the Bills list and the bill page; the bill card on MAPLE's bill pages draws its own bar and keeps its title. Verified: yarn test:website passes; check-types shows only the 4 known errors; prettier passes. Not verified: the tooltip in a browser, on any page; tapping a segment on a phone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The years under the Key figures heading ("2025–2026") were lighter and
gray. They now take the heading's weight, spacing and color, and keep
only their own line.
Verified: screenshot of the panel at 1280 px against yarn dev;
check-types shows only the 4 known errors; prettier passes. In the same
session, hovering a position bar segment on the Overview showed its
tooltip ("Support: 133 organizations") in about 70 ms, which 5112f0a
lists as not verified.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Bills list's panel said "select a column heading to sort", but only the Organizations column sorts. The panel now gives just the bill count. The Clients list keeps the hint, so its string stays. Verified: on yarn dev, the Bills panel heading reads "5,660 bills with reported lobbying" with no hint; check-types shows only the 4 known errors; prettier passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The boxed bar holding the session select and the search box on the Bills, Clients and Lobbyists pages was as wide as its filters, so it ended short of the list's panel. A boxed bar now spans the list and its search box takes the extra room (wrapping to its own line on phones). The Overview's bar, with only the session select, keeps fitting its filters (boxed="fit"). Verified: on yarn dev, the bar's and the list panel's edges match on all three pages at 1280 and 375 px (82-1198 and 12-363), and the Overview's bar still fits its select; check-types shows only the 4 known errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A lobbyist's page kept the older look: a compact Bootstrap table on the gray background whose Period column was cut off, and a right column of plain lists, 21c's clients alone running past 100 lines. The Lobbied bills table now sits in a panel with the activity count and its pagination, in the Lobbyists list's table style (a style of this page's own; the bill card's shared table is unchanged). It scrolls inside its panel on phones. Each right-column group is its own panel. Clients collapse to their name and total, expanding to what each paid by period; Registrations and Disclosures show the latest three, then Show all / Show fewer. NonBillActivity moves to its own file, and the shared filings table drops the bill and amount columns its one caller, the bill card, never showed. The ▸/▾ button style is shared with the activity tables. Verified: on yarn dev, 21c at 1280 and 375 px (table fits its panel at 1280 and scrolls at 375; a client expands to its periods and collapses; Registrations shows 3, all 12, then 3); Aaron Agulnek's panels; Aaron Michael Vega's files-through-employer note inside the Bills panel; yarn test:website passes; check-types shows only the 4 known errors. Not verified: the bill card on MAPLE's bill pages in a browser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The six Lobbying Explorer pages have been revised:
The pages read the aggregates that the scraper precomputes, the "No client" placeholder is listed as such rather than as an organization, and lobbyist totals are exact. To keep page data light, the long lists embed only the current session and load others through cached API routes, and every page regenerates daily. The list pages read their rows from the field the stats run now writes them under to stay within Firestore's index entry limit.
Closes hyphacoop#89
Closes hyphacoop#90
What's in it
Deploying
The redesigned pages read data that only exists once prod has the new scraper and a reparsed archive. So prod's data goes first, and this PR merges only after that. Dev already went through steps 1 and 2.
1. Release
maintoprod(scraper, indexes and rules)main.rowsindex exemption infirestore.indexes.json, whichdeploy-prod.ymldeploys. It also ships everything else merged tomainsince 10/15: conferences, the finance backfill, bill summaries and more.proddoesn't have the "Deploy the lobbying scraper" workflow yet (Lobbying scraper: rebuild the image on merge #2309), so this release is the first time it runs for prod. It uses theprodenvironment'sGCP_SERVICE_ACCOUNT_KEYand only updates the image of an existingmaple-lobbying-scraperjob.lobbying-scraper/deploy.sh digital-testimony-prodfrom theprodcheckout. It creates the job if it's missing, sets 2 GiB and 1 vCPU (the stats step runs out of memory at the 512 MiB default, and the workflow doesn't change memory), and sets up the weekly trigger. Then check that the job hasARCHIVE_RAWand noARCHIVE_DIR(ingestion doc, "Deploying the Cloud Run Container").prodpush.2. Backfill prod
These are the steps in the ingestion doc's "Backfilling Newly Parsed Fields", run with
--project digital-testimony-prod. The rehearsal on dev is in hyphacoop#106 (see the implementation notes).gcloud firestore export --collection-ids=…). The cleanup phase deletes docs, so this export is the only way to undo the backfill.refetch-errors: dry run, then--execute.write: expect "superseded" > 0 andsuperseded_without_current0.registrations.cleanupforfilingsand forregistrants. Read the dry runs before running--execute. Stale filings are expected only for superseded versions; anything else means doc ids changed, so stop.filing-periods: expect "updated" 0.gcloud run jobs execute maple-lobbying-scraper --args=--mode,stats --region=us-central1 --project=digital-testimony-prod, once step 1's index deploy has finished.clientList_*,clientList_all,firmList_*) should have onerowsfield. Prod can be a little ahead of dev from newer filings.The site that's live now should keep working on the backfilled data. The dev site has run
main's current Lobbying Explorer on dev's backfilled data since hyphacoop#106, so check https://maple-dev.vercel.app/lobbying before starting.3. Merge this PR into
mainThe dev site then shows the redesigned pages on dev's data. Check
/lobbyingand each list, Bill and Client page there, at phone width too.4. Release
maintoprodagainThat's the next "Deploy to PROD" PR. The lobbying pages are built at deploy and then regenerate daily, so they show prod's data from the first load. Check https://mapletestimony.org/lobbying and the same pages as in step 3.
Rollback: reverting this PR and releasing restores the old pages, which keep working on the backfilled data (see step 2). Undoing the data changes means importing the export from step 2.
🤖 Generated with Claude Code