Repository navigation
[Feature] Pay for shop orders with an OpenBanking card; cross-app purchase tasks - #86
Open
aaronsmulktis wants to merge 29 commits into
Open
aaronsmulktis wants to merge 29 commits into
aaronsmulktis wants to merge 29 commits into
Conversation
`git worktree add .claude/worktrees/<topic>` puts a full checkout inside the repo. It showed up as untracked in the main checkout, so a stray `git add -A` would have committed the entire nested tree.
The shop was a port of Princeton's WebShop that needed OpenJDK 21 and a
~1000-product dataset downloaded from Google Drive by `setup.sh`. Products
were searched through a Lucene index built by pyserini, which is a JNI
binding and hence the JDK. None of that reaches an offline eval node, so the
app shipped disabled and could not be launched from a clean checkout.
Three things made the rewrite cheap. `get_reward`/`get_goals` -- the whole
WebShop scoring model, including its spaCy POS-tagging and thefuzz attribute
matching -- were imported but never called. No task in config/tasks targets
the shop. So nothing depended on the dataset's annotations or on the state
shape, and the schema was free to change.
* Catalog seeds from `config/apps/onlineshop/content/` like every other app
(40 products, 8 categories), so `content` is a real variation axis.
* Search is SQLite FTS5 with its built-in `bm25()`, weighted title > options
> bullets > description. Option values are indexed, matching the document
the original built for Lucene. No JDK, no new dependency.
* Persistence is one SQLite file of ordinary relational tables. Order lines
were keyed by a stringified Python tuple ("('B07YPYJ32Z', '{}')") read back
with ast.literal_eval; `/onlineshop_all` now returns plain nested JSON.
* Rendering is FastHTML on the shared design tokens, so `apps/theme=` applies,
plus a new `layout` group: default, grid, compact_table.
* Product imagery is a generated SVG swatch keyed on the sku. The old cards
pointed at Amazon CDN URLs, which is egress the eval nodes do not have.
Enabled by default now that it has no setup step. Drops pyserini, spacy,
thefuzz, gdown and faiss-cpu (the last two already unused) -- 33 packages
including pyjnius. setup.sh keeps only the Java 21 install, moved to the map
app which is the actual OpenTripPlanner consumer.
Test plan: uv run -m pytest tests/ -> 772 passed, 6 skipped (was 724/7; the
onlineshop test no longer skips on missing Java). tests/test_onlineshop.py
adds 47 covering search ranking and FTS operator injection, cart merge/split
on options, checkout, the reward state shape, all three layouts, per-app and
global themes, german content, and absence of external hosts.
Not verified: reference screenshots under tests/generated_screenshots for the
onlineshop_* routes still show the old Bootstrap UI and need regenerating;
Chromium would not start in this sandbox.
Three things found while using the shop: * Cart lines showed a unit price and a quantity but no line total. Added "$89.00 each x 3 = $267.00". * Two lines of the same product differ *only* by their options, which were rendered as a small muted caption -- so a cart holding a fine-nib and a broad-nib pen read as duplicated rather than as two variants. Options are now chips, and a deselected line says so. (The merge itself was correct: identical options merge in every path -- urlencoded, multipart, in-process.) * No way back to the landing page from the cart or an item. Added a "Shop" link to the shop bar, plus a cart count. It lives there rather than on the header logo because `clickable_logo` defaults to false and is a start-page variation axis; navigation must not depend on a difficulty knob. Also name the tables explicitly. fastlite derives them from the class name and singularises, so `Order` became `order` -- a SQL reserved word, breaking `SELECT * FROM order` for anything reading the database directly. That defeats the point of the schema, which is to be readable by other apps and by humans. Now products / cart_items / orders / order_items, as documented. Test plan: uv run -m pytest tests/test_onlineshop.py -> 51 passed (4 new, covering line totals, option chips, per-page navigation, and table naming). Verified against a live launch with sqlite3: unquoted joins across all four tables now work.
The installation page still described the old shop as an optional extra requiring OpenJDK 21, a gdown dataset and setup_pyserini.sh. Replace that with what is actually true -- the shop needs no setup and is on by default -- and add the parts a user needs after launching: * how to enable/disable it and vary layout, content and theme; * how to find the SQLite database for the run they were just clicking through (`ls -t log_outputs/*/databases/onlineshop.db | head -1`) and query it; * a short architecture note: catalog seeded from Hydra content, FTS5 + bm25() for search, four relational tables, FastHTML on the shared design tokens, generated SVG imagery, `/onlineshop_all` as the reward surface. Kept a collapsed note on the old JDK/Lucene setup, since the Google Drive links it used are now dead and anyone following an older guide will get stuck. Added a matching section to the README. Verified every documented override composes (layout x3, content x3, theme global and per-app, enable, products_per_page) and the sqlite3 snippets run against a live launch.
Thumbnails were a coloured square with the product's initials, which made a listing scan like a spreadsheet. Replace with 35 line-art glyphs -- headphones, skillet, tent, fountain pen -- picked by keyword from the title with a per-category fallback, so a product added to the config without a matching keyword still gets something appropriate rather than a blank. Still generated rather than fetched, for the same reason as before: the original hotlinked Amazon CDN URLs, which eval nodes cannot reach and tests/test_no_egress.py forbids. Deterministic, no dependency, ~2kB each, and hue stays keyed on the sku so a product looks the same on every page. Keyword order is load-bearing and was wrong on first pass: "LED Desk Lamp" matched the standing-desk entry because it contains "desk". Ordering is now commented and pinned by tests, along with "Desk Organizer" and the "Backpacking Tent" / "backpack" collision. Test plan: uv run -m pytest tests/test_onlineshop.py -> 62 passed (11 new). Audited all 40 products: 34 distinct glyphs, zero falling through to the category default.
Replace the generated product catalog with an optional importer for the WebShop item dump, so no model-written product copy ships in the repo. `scripts/fetch_webshop.py` downloads the dump from its HuggingFace mirror and converts it into a `webshop` content pack. The output is gitignored: the records are scraped Amazon listings, so they stay on the machine that downloaded them rather than being redistributed from here. Field names are read from candidate lists because the dump has been re-exported several times -- the current one uses name/full_description/small_description/ pricing -- and `--inspect` prints the real schema when they drift again. Image URLs are dropped on import, including the ones hidden in `customization_options` sibling keys, and the script aborts if any survive. `content/default.yaml` is now chrome only. With an empty catalog the start page leaves the shop unregistered and drops its tile, so the app is absent rather than an empty storefront. `content/fixture.yaml` carries the small mechanical catalog the tests need. Also fixes two things real titles exposed: * Glyph keywords matched as bare substrings, so "pen" fired on "Open Toe Sandal" and "tent" on "content". They now match whole words with an optional plural, and the table is widened against real titles: keyword coverage goes from 41/200 to 113/200, the rest taking a category default. * WebShop's coarse `category` field put nearly everything in one slug, so the mapping reads the `product_category` breadcrumb deepest-crumb-first. `scripts/render_glyph_sheet.py` renders a catalog's line art to one page, tinting the products that fell back, so coverage can be eyeballed without launching the app. Test plan: 799 passed, 6 skipped. Verified against the real dump -- 47 pages over 200 scraped products reference no external host and the seeded database holds no URLs. ruff clean on both scripts; mkdocs build --strict clean (site/ left unbuilt).
Keep the source catalog's image URLs and render them when asked, instead of discarding them at import. `apps.onlineshop.product_images` picks between `glyphs` (default) and `hotlink`. Under `hotlink` a product with several images becomes a CSS-only carousel -- one hidden radio per slide, one clickable dot per image -- so the controls are real elements an agent can click with no JavaScript and no new dependency. A single image renders without controls, and a product with none falls back to its glyph. The glyph also ships inside every carousel as the fallback: each <img> has an onerror that reveals it, so a failed fetch degrades to line art rather than a broken-image icon. One failure takes the whole carousel down deliberately -- the common case is "no network", where all of them fail, and a half-populated carousel is a worse observation than a consistent one. Default is `glyphs`, and an unrecognised value falls back to it, because hotlinking is only safe when the network is really there: * the eval nodes have no outbound network, so the images do not arrive while the page still returns 200 and a screenshot-scored agent gets graded on an observation that quietly lost its imagery; * TestNoEgress fails under `hotlink`, by design; * for the WebShop catalog the URLs point at Amazon's CDN. The importer's URL guard is now field-scoped rather than whole-record: text fields still must not contain a URL, since those render under either mode, while `images` is the one place a URL is legitimate. Amazon's transparent spacer gif is filtered out of the image list, and images are capped at four to match the carousel dot rules. Test plan: 809 passed, 6 skipped. Verified both modes over the real 200- product catalog -- `glyphs` emits no product-media markup and references no external host, `hotlink` emits 34 slides against m.media-amazon.com. Browser rendering unverified: chromium crashes in this sandbox, so the carousel is checked at the markup level (checked-radio adjacency, unique radio group per sku, glyph sibling present) rather than visually.
Six app modules imported their siblings through `src.open_apps...`, which only resolves because `open_apps/__init__.py` pushes the repo root onto sys.path on package import. That shim works, but it means the same module is loaded twice under two names -- `src.open_apps.theme` and `open_apps.theme` are distinct objects with separate globals -- so identity checks and any module-level state silently diverge depending on which spelling a caller used. Import them the way the package is installed instead. The shim stays: it is a public compatibility path and something outside this repo may still use the old spelling. Also guard `render_theme_css` against a theme token containing `<` or `>`, which could otherwise close the <style> block it is rendered into. No shipped theme has such a value, so this is defensive only -- the neighbouring checks already cover the property name and the @import URL. Test plan: 809 passed, 6 skipped. Confirmed no module remains loaded under the `src.` name after importing the apps, and that the theme guard drops an injected `</style><script>` value while leaving quoted font stacks intact.
The shop needed `scripts/fetch_webshop.py` to run before it would appear at all, which made a fresh clone a two-step setup and left the storefront absent rather than broken if you skipped it. Check the generated pack in instead: `apps/onlineshop/content=webshop` is now the whole switch. - un-gitignore config/apps/onlineshop/content/webshop.yaml (200 products) - the pack sets `product_images: hotlink`; the shop-wide default stays `glyphs`, so only runs that select this catalog reach the CDN - add an explicit `_self_` to apps/onlineshop/default.yaml so a content pack can override an app-level key at all -- without it Hydra composes the app defaults last and the pack's `product_images` was silently discarded - fetch_webshop.py becomes a regenerator rather than required setup, and its header round-trips what is committed - attribution in README/docs now says the data is redistributed here, because it is
999 products, up from 200 -- the whole of items_shuffle_1000.json less one record with an empty title, which `convert` skips. Search, pagination and the category strip all have something to work with now. `--limit` defaults to 1000 to match, so a bare `fetch_webshop.py` rebuilds the committed pack rather than silently shrinking it to 200. Product counts and the glyph-sheet sample output in the docs are regenerated against the real catalog (531 titles match a glyph keyword, 468 fall back to the category).
Alignment. The search field had no `flex`, so it kept its intrinsic ~150px while its form grew to 1020px -- the slack opened as dead space between "Search" and "Cart" rather than widening the field. It now grows (capped at 34rem) and cart/orders anchor right. The cart's three action forms sat in an unclassed div and stacked as ragged blocks; `.cart-actions` makes them one end-aligned group on a shared 7rem button width. The buy row's number input rendered 9px shorter than its button, so both control rows align on `stretch` instead of `center`. The footer link moves to its own row, where it no longer shares a line with a page's closing CTA at a different margin-top. Measured in chromium at 1280/900/600px: every control in the bar is 30.7px tall, the three cart buttons share an exact left and right edge at all three widths, and the buy row's input and button now match at 30.7px. Java. The shop's search dropped pyserini/Lucene a while back, but the JDK story was still told in nine places, and `appserver.registered_apps` claimed `onlineshop` needs Java 21 installed, which is simply false. Removed from the shop's docs, config and docstrings. The map app's OpenTripPlanner JDK is a real dependency and is untouched.
`uv run launch.py apps.onlineshop.enable=True` did not produce a shop: the flag was already the default, and what was actually missing was a catalog. The default `content` pack was `default`, which is chrome with `products: []`, so the start page left the routes unregistered. Committing the catalog was supposed to remove that setup step; leaving it unselected kept it. `content: webshop` is now the default, so a bare launch has a 999-product storefront. `content=default` remains the chrome-only pack as an explicit choice, and the gate still hides the shop when a pack has nothing to sell. This makes the default configuration reach the network, since the webshop pack carries `product_images: hotlink`. That was invisible to the egress suite -- its ROUTES list has no `/onlineshop` -- so `/onlineshop` joins the ratchet with `m.media-amazon.com` as its one allowed host, the only content-driven entry among styling dependencies. Eval runs on a node without egress want `apps.onlineshop.product_images=glyphs`. Three tests asserted absent-by-default and now assert the inverse; the empty-pack path they covered is retained against an explicit `content=default`.
…grid The shop was opt-in on the openbanking branch and is enabled by default on the shop branch, so the gate has no remaining caller after the rebase.
Cards gain `card_cvv`, masked on the card face and revealed by the same click that reveals the PAN -- a checkout elsewhere needs it, so it has to be reachable, but it should cost the agent the extra click. `authorize_card_purchase` is the app's first write path. It matches every card detail, approves inside `available_credit + overlimit_grace` ($10 by default), debits the card and posts the charge, plus a second row naming the overage when the charge used the grace band. Rows post pending so no wall-clock value reaches /openbanking_all; declines say nothing about which field was wrong, or the CVV is three digits of brute force away. set_environment now clears before re-seeding, like the shop's does -- a reset has to put the card's figures back rather than collide on the seeded keys.
Checkout's card step was cosmetic: a brand dropdown checked against a config allow-list, with no number, no expiry and nothing behind it. It now asks for the PAN, expiry, CVV and cardholder name and authorizes them against a card held by OpenBanking, which is where the available credit lives. The apps already share a process -- start_page imports each and merges the route tables -- so the shop calls the bank directly rather than going back out over HTTP to its own server. The import is lazy and guarded: a run without the bank declines cleanly instead of raising out of a checkout handler. The charge happens before the order is written, because an order with no matching authorization is the one state a checkout must not leave behind. A decline redirects back with the issuer's message and leaves the cart intact. Orders record the paying card's last four, and /onlineshop_all reports it. Defaults to on: `allowed_credit_cards` now lists the brand the seeded card actually carries, so the check is exercised rather than skipped.
Checkout has been able to charge a card since 020778d, but nothing measured it: every task in config/tasks/openbanking.yaml reads a figure off the ledger and writes the answer into todo/calendar/messenger, because the bank was read-only. BuyWithCardTask is the other shape -- the scorable delta lands in the shop and the bank at once, which is the first time either app's state has had to move together. What it measures is whether an agent can carry a value across apps. The PAN, expiry and CVV are only on the card's page in OpenBanking, each behind the click that unmasks them, and the checkout form takes no order without all three. The target is a new order carrying the paying card's last four plus a pending Card row that debits the headroom by exactly the order total, so buying the right thing on the wrong card still fails. A checkout writes three things no target can predict: a random order id, a wall-clock date, and a ledger description embedding that order id. Dropping the description would throw away the only thing tying a charge to the order that caused it, so instead every order id in the shop's slice is substituted with a fixed token on both sides -- the diff still asserts the charge names an order that exists. Row id and position go too; both are functions of the bank's insertion scheme rather than of what the purchase did. The normalization is opt-in, like reply_contacts. Every other task should keep failing on a spurious order, ids and all, and CompositeTask turns it on itself when a sub-task buys. A purchase needing the overlimit grace refuses to build a target rather than score half a ledger it did not describe. Two tasks ship with it: the purchase alone, and a composite that chains it with a todo naming the headroom left afterwards -- a figure that does not exist until the charge posts, so the agent has to cross back. The tests score against states taken from the running apps rather than a fixture, since the failure worth catching is the target drifting from what a checkout actually writes. That is also why the two shipped tasks sit out test_openbanking's generic self-consistency sweep: states/initial_state.json predates both apps, and refreshing it would move the ground under every other task in the repo. The same assertion runs in test_buy_with_card.py instead.
1. update GPT 5.5 model 1. update banking tasks 1. add buy_the_mirrors_just_over_the_card_limit 1. add attempt_the_copier_far_over_the_card_limit 1. record purchase declines
shop-with-bank already carries rebased copies of every webshop-rewrite commit on top of the theme + layout refactor, so each conflict resolves to this branch's side. What the merge adds is main's dev server (dev.py, scripts/dev.sh) and its README notes (dev server, reproducing the paper). docs/index.md is kept as-is: the auto-merge re-added the pre-refactor Layout section a third time.
- ModelArgs.make_model raises a ValueError naming the fix when hostname is null, instead of building http://None:8000/v1 and failing three minutes later as a bare RetryError - AgentLauncher records errored / err_msg / stack_trace in the W&B summary, so a crashed episode no longer looks like a zero-reward run
Serves a self-hosted agent model with --host 0.0.0.0, caches weights under /checkpoint, clears inherited proxy vars, and writes the node name to ~/logs/vllm_host_<jobid>.txt. Account/QOS/partition are placeholders.
meta-cla
Bot
added
the
CLA Signed
aaronsmulktis
marked this pull request as ready for review
October 13, 2026 13:38
aaronsmulktis
changed the title
main's TestGemmaCoordsConfig builds a self-hosted agent without a hostname, which this branch's fail-fast now refuses. The tests build the client but never call it, so any name will do.
aaronsmulktis
changed the title
…xpiries A refusal for insufficient credit now posts one zero-amount DECLINED row on the card, so a refused checkout is distinguishable from a cart that was filled and never submitted. DeclinedCardPurchaseTask scores that row alongside the cart line. Detail typos still write nothing. Expiry is parsed as a real month/year and judged against the bank's pinned as_of_date; checkout refuses a past or malformed expiry before asking the bank, and an expired card on file declines.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lets the online shop take payment with a credit card held in the OpenBanking app, and adds tasks that are scored on both apps at once.
Until now every OpenBanking task was read-then-act. The bank was read-only, so the agent looked a figure up and the scored change landed in todo, calendar or messenger. With this PR, checkout charges a card through a single write path into the bank. A purchase is then a change in two apps' state: the shop gains an order, and the bank gains a pending charge that lowers the card's available credit by exactly the order total.
The cross-app step is real, not a form fill. The card number, expiry and CVV appear only on the card's page in OpenBanking, each masked until clicked, and checkout refuses an order without all three.
uv run launch_agent.py agent=GPT-5.5-computer-use tasks=openbanking \ task_name=buy_the_console_table_with_the_business_card \ browsergym_env_args.max_steps=40What's in it
OpenBanking: cards can be charged (
openbanking_app/main.py,config/apps/openbanking/)authorize_card_purchaseis the bank's only write path. It matches the card number, expiry, CVV and cardholder name against a seeded card, and refuses a card whose expiry has passed. It then checks the amount againstavailable_creditplusoverlimit_grace(default10.00). There are three outcomes:Cardrow and lowers available credit.OVERLIMIT NOTICErow naming the overage.DECLINEDrow naming the merchant and the attempted amount (DECLINED - OPENAPPS SHOP $21,499.95 - insufficient available credit). No money moves and the balances stay the same. Real card activity feeds show declined authorizations the same way. Without the row, a refused checkout left both apps exactly as they were before the attempt.position, so they sort above the seeded ledger without renumbering it.09/29,9/2029,0929, …), so13/29is rejected. Expiry is judged againstas_of_date(default2026-08-26, the seeded statement cycle), not the wall clock, so the seeded09/29card can't expire under a task and turn every purchase into a decline. Setas_of_date: nullto use the real date.Online shop: checkout uses the bank (
onlineshop_app/main.py,config/apps/onlineshop/default.yaml)enable_credit_card_checknow defaults totrue. Checkout asks for card number, expiry, CVV and cardholder name, and authorizes against the bank.allowed_credit_cardsis matched against the stored card'scard_brand, not anything the shopper types. Ledger rows are labelledOPENAPPS SHOP {order_id}(card_descriptor), so each charge traces back to its order.card_last4. The order card shows a "Paid with" row.Tasks and rewards (
tasks/tasks.py,config/tasks/openbanking.yaml,docs/tasks.md)BuyWithCardTaskbuilds the target state: an order carrying the card's last four, plus the matching bank charge, plus the overlimit row when the total falls in the grace band.DeclinedCardPurchaseTaskhandles a checkout the app refuses. It scores two things:DECLINEDrow. This fails an agent that filled the cart and never submitted checkout, which used to leave identical state, while the shortfall the todo asks for could be worked out from the cart total and the card page.A second refused attempt posts a second row and fails, like a duplicate message would.
AppStateComparisongainsnormalize_purchases. It ignores the random order id, the wall-clock date, the order id embedded in the ledger row, and ledgerid/position. It's opt-in, so every other task still fails on a stray purchase. Both purchase task classes use it, andCompositeTaskturns it on when a subtask checks out.Both task classes refuse to build a target for a total the app would treat differently, for example a purchase priced into a decline. Mispricing raises an error instead of scoring something the app can't produce.
Four new tasks in
tasks=openbanking:buy_the_console_table_with_the_business_cardbuy_the_console_table_and_log_the_chargebuy_the_mirrors_just_over_the_card_limitattempt_the_copier_far_over_the_card_limitDECLINEDrow; the agent logs the shortfall as a todoEval infra
vLLM_agent.pyfails at startup whenagent.hostnameis missing. Previously it builthttp://None:8000and surfaced a bareRetryErrorthree minutes later.launcher.pywriteserrored,err_msgand a truncatedstack_traceto the W&B run summary. A crashed episode no longer looks like an agent that ran and scored 0.scripts/serve_vllm.sbatchkeeps a vLLM server running forconduct_slurm.shto find. Account, QOS and partition are placeholders.The branch was merged with current
main. Conflicts were in the shop checkout (it keepsmain's new layout, with the card fields inside it),launcher.py,theme.pyanddocs/inspecting_databases.md.Reviewer notes
uv run launch.pyshop now asks for card details at checkout. No existing task checks out (the only shop task isnavigate_to_onlineshop), butapps.onlineshop.enable_credit_card_check=falserestores the old behaviour for anyone who relied on it.max_steps: the default is 10, which is too low for these tasks. Use about 40.unit_pricein each task has to match the catalog being served (content=webshopby default), because the reward can't look prices up. The task'sdescriptor,charge_type, overlimit and declined strings follow the bank's content pack, so a german or mandarin run needs them overridden.config/agent/GPT-5.5-computer-use.yaml:model_namechanges fromgpt-5-5-genai-responsestogpt-5-genai-responses.Test plan
test_buy_with_card.py(rewards across all three outcomes, normalization, composites),test_card_payments.py(authorization),test_vllm_model_args.py(hostname check).content=webshop(copier ×5, real card): the declined checkout adds theDECLINED - OPENAPPS SHOP $21,499.95row. With the shortfall todo, "cart filled, never checked out" scoresFalseand "declined checkout" scoresTrue.test_openbanking.py,test_onlineshop.py,test_no_egress.py.TestGemmaCoordsConfig, wheremain's new test builds a vLLM agent with no hostname and now trips this PR's hostname check. The test now passes a placeholder hostname; that file andtest_vllm_model_args.pypass on rerun.buy_the_console_table_with_the_business_cardwithagent=GPT-5.5-computer-use,record_video=Trueand W&B runs through and is scored. GPT-5.5 did not complete the purchase on that run (scoredfail).