ride-static

Static-site generator for ride.i-d-e.de — RIDE. A review journal for digital editions and resources, published by the Institut für Dokumentologie und Editorik (IDE).

The pipeline reads the TEI XML review corpus under issues/{N}/reviews/, the editorial pages as TEI under pages/ (with a Markdown fallback under content/), and one metadata.yaml per issue. A single GitHub Actions workflow produces a complete site/ tree with per-review HTML and PDF, aggregation pages, a Pagefind index, OAI-PMH and JSON-LD interfaces, a sitemap, three syndication feeds (Atom, RSS 2.0, RSS 1.0/RDF), and a redirect layer that keeps the old WordPress URLs and feed paths working. The output is fully static; no runtime server, no database, no per-request work beyond serving files and the client-side search.

It replaces the previous eXist-based dynamic site. Written in Python with Jinja templates. Every script and parser module ships with pytest coverage; integration tests drive off the in-repo TEI files (Real-Corpus-Drive).

Workflow overview

                ┌────────────────────────────────────┐
   inputs       │  issues/{N}/reviews/*-tei.xml      │   (canonical content)
                │  issues/{N}/metadata.yaml          │   (editorial metadata per issue)
                │  schema/{ride.odd, ride.rng}       │   (validation contract)
                │  pages/*.xml  (ride-pages.rng)     │   (editorial pages as TEI)
                │  content/*.md, content/home/*.md   │   (Markdown fallback, home widgets)
                │  config/{element-mapping,nav}.yaml │   (presentation + navigation)
                │  ../ride/issues/.../pictures/      │   (figure assets — still external)
                └─────────────────┬──────────────────┘
                                  │
                ┌─────────────────▼──────────────────┐
   scripts/     │  Discovery (inventory, structure,  │   one-off introspection
                │  sections, ids, refs, taxonomy,    │   → inventory/*.json
                │  odd_extract, p5_fetch,            │   → knowledge/data.md
                │  cross_reference, render_*)        │     knowledge/schema.md
                └─────────────────┬──────────────────┘
                                  │
                ┌─────────────────▼──────────────────┐
   src/         │  parser/    TEI → Review domain    │   immutable dataclasses
                │  render/    Review → HTML/PDF/JSON │
                │  build.py   end-to-end CLI         │
                │  validate.py  RelaxNG pre-build    │
                └─────────────────┬──────────────────┘
                                  │
                ┌─────────────────▼──────────────────┐
   outputs      │  site/                             │   uploaded to GitHub Pages
                │   ├─ issues/{N}/{id}/{index.html,  │     by .github/workflows/build.yml
                │   │            review.pdf, *.xml}  │
                │   ├─ tags/, reviewers/, resources/ │
                │   ├─ api/{corpus,build-info}.json  │
                │   ├─ oai/, sitemap.xml             │
                │   ├─ pagefind/                     │   Pagefind client-side search
                │   └─ static/{css, js, fonts}       │
                └────────────────────────────────────┘

Editorial workflow

Reviews are prepared in the private companion repository i-d-e/ride-editors. Its convention: one folder per issue (issue-{name}/), inside it one folder per review ({slug}/) holding the TEI file {slug}-tei.xml, the figure images in pictures/, and the article image {slug}-wordcloud.png, next to working material (submissions, versions, peer review). Publishing a review means carrying these three pieces over, as described below.

Publish a review (from ride-editors)

1. Finish the TEI in ride-editors. Figure references keep the canonical URL form https://ride.i-d-e.de/wp-content/uploads/issue_{N}/{slug}/pictures/{file}; the build rewrites them to the static site at build time. The <fileDesc> carries the issue number:

   <seriesStmt>
     <biblScope unit="issue" n="22"/>   <!-- ← matches issues/22/ -->
   </seriesStmt>

2. Copy the TEI file into this repository at issues/{N}/reviews/{slug}-tei.xml. The directory number and biblScope @n must match — the build stops with a clear error if they disagree.
3. Copy the images into the picture repository i-d-e/ride at issues/issue{NN}/{slug}/pictures/ (two-digit issue number on disk, e.g. issue05, issue22). The pictures are the only part that still lives outside this repository, because of their size.
4. Copy the wordcloud into this repository as static/images/wordclouds/{slug}.png (or .jpg) — same file as in ride-editors, renamed to the bare slug. A missing wordcloud is not an error; the issue page simply shows no thumbnail for that entry.
5. Optionally check locally: python -m pytest tests/test_validate.py and python -m src.build. Then commit and push — CI rebuilds and deploys the whole site. An issue grows review by review (rolling release); every push republishes everything, so nothing can go stale.

Add a new issue

1. Create issues/{N}/metadata.yaml:

   issue: '23'
   title: 'Issue 23: Scholarly Editions'
   doi: 10.18716/ride.a.23
   status: regular              # or "rolling" for open volumes
   publication_date: 2026-09
   description: |
     Free-text intro shown above the contribution list.
   editors:
     - name: Ulrike Henny-Krahmer
       affiliation: Universität Rostock
       orcid: https://orcid.org/0000-0003-2852-065X
     - name: Finnja Borchardt
       role: assistant
   contribution_order:           # optional — fixes ordering on the issue page
     - ride.a.23.1
     - ride.a.23.7

   Only issue: is required. Schema (src/render/issues_config.py): title, doi, status (regular/rolling), publication_date, description, editors[], contribution_order[]. Typos in field names break the build (no silent fail).
2. Add the first reviews under issues/{N}/reviews/ as above.
3. The home page and issue-overview pages pick the new issue up on the next build; no further wiring needed.

Preview before publication

How unpublished reviews are shown to authors and the IDE for final checking is an open editorial decision; the options (password-protected hosting vs. publicly built but unlisted pages) and a concrete proposal are written up in the staging section of knowledge/pipeline.md. Until that is decided, a preview is generated locally. Copy the draft TEI into issues/{N}/reviews/ in a local working copy (without committing), then

python -m src.build --pdf
python -m http.server -d site

shows the complete site, draft included, at http://localhost:8000/.

Edit an editorial page (Editorial, Imprint, Criteria, …)

Editorial pages, everything outside the reviews and factsheets, are TEI. Each lives at pages/{slug}.xml, or at pages/{section}/{slug}.xml to carry a navigation section in its URL. They validate against the page profile schema/ride-pages.rng, a deliberately small grammar with a reduced teiHeader and a body of div/head/p/list/table/eg blocks plus ref/persName/email/hi/code/lb inline. The build parses each file into the Page model (src/parser/page.py → src/model/page.py) and renders it through src/render/page.py into the shared single-column editorial.html template. The deployed build renders these with precedence over the legacy Markdown.

The URL mirrors the file location (URL scheme v2). A file directly under pages/ keeps a flat URL (pages/criteria.xml → /criteria/); a file in a section folder takes that prefix (pages/about/team.xml → /about/team/), matching its navigation section. See docs/url-scheme.md.

1. Edit the body of pages/{slug}.xml (or pages/{section}/{slug}.xml). Stay within the elements ride-pages.rng allows, or tests/test_pages_schema.py fails.
2. Add a new page: drop a new file under pages/ that validates against the profile. Put it in a section folder (pages/about/{newslug}.xml → /about/{newslug}/) or at the top level for a flat /{newslug}/; discover_pages() finds it automatically. Add an entry to config/navigation.yaml, and tests/test_render_navigation.py checks that every menu URL resolves to a built page.
3. A few generator-native pages stay Markdown under content/: the About overview and the data charts/questionnaire pages (these are data-driven, not prose). For any slug no TEI page covers, content/{slug}.md renders as fallback. Pass --no-tei-editorials to build the Markdown set instead.
4. Home-page widgets live in content/home/*.md (frontmatter pattern). Global navigation is config/navigation.yaml; the loader validates every entry, so a typo breaks the build.

Adjust TEI-to-HTML rendering

For most changes, edit config/element-mapping.yaml — it binds TEI elements to template paths and CSS classes without touching Python. Schema and contract are in docs/extending.md.

For new TEI elements or new domain shapes, see docs/extending.md (six files to touch: parser, dataclass, mapping, template, CSS, test).

Build workflow

Locally

python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt

python -m pytest tests/                             # full suite, run from repo root
python -m src.build                                 # full build → site/
python -m http.server -d site                       # local preview at http://localhost:8000

--pdf, --linkcheck, --base-url and the remaining build flags are documented in full in knowledge/pipeline.md.

python -m src.build runs three stages in order: validate (src/validate.py, RelaxNG against schema/ride.rng), parse (src/parser/, TEI into immutable Review dataclasses; classifies references, copies figures from ../ride/issues/.../pictures/ if the sibling repo is present), render (src/render/, HTML + optional PDF + aggregations + OAI-PMH + JSON-LD + sitemap + corpus dump + redirects).

Each run records site/api/build-info.json with commit hash, corpus version, validation findings, asset report, and licence — the build is reproducible.

Discovery scripts

Stage-0/1 introspection lives in scripts/. Each script exposes run(...) for testing plus a main() that writes JSON to inventory/ (gitignored) or Markdown to knowledge/. Full list and outputs: see CLAUDE.md. They run as part of CI but are not required for the build itself.

Deploy workflow

Single workflow at .github/workflows/build.yml. Triggers on push to main (TEI / content / code paths), on manual workflow_dispatch, and on a notification from a companion repository (GitHub repository_dispatch): a push to i-d-e/ride (picture updates) or i-d-e/ride-editors (work in progress) can rebuild this site, because those pushes do not reach this repository on their own. The notification needs a small one-time setup per companion repository — one workflow file plus one access token; copy-ready templates and instructions are in docs/upstream-workflows/.

The workflow checks out this repo plus i-d-e/ride for picture assets, installs Python + WeasyPrint system libs, runs pytest and the discovery scripts, runs python -m src.build --pdf, builds the Pagefind index, and deploys site/ to GitHub Pages.

The second checkout is the only remaining external dependency; it can drop once the ~437 MB of picture assets migrate into this repo (Git-LFS likely needed).

Further reading

• CONTRIBUTING.md — setup, hard rules, conventions, and the pointer table to internal docs.
• CLAUDE.md — repository layout, script outputs, project conventions.
• docs/extending.md — adding a TEI element or render variant.
• docs/url-scheme.md — versioned URL contract.
• knowledge/ — Obsidian-style vault with its own index (knowledge/INDEX.md): corpus reference (data.md, schema.md), design intent (architecture.md, pipeline.md), product specification (specification.md, interface.md). Cross-references use [[wikilink]] notation.
• knowledge/journal.md — session-by-session decisions and current entry point.

Per-directory READMEs describe their own folder:

• scripts/README.md — Stage 0/1 discovery scripts and their JSON/Markdown outputs.
• src/README.md — parser, model, render, and build modules with the data flow.
• config/README.md — navigation.yaml and the spec-only element-mapping.yaml.
• static/README.md — css, js modules, fonts, and vendored image assets.

Status

Live: per-review HTML and PDF, aggregation pages (tags, reviewers, resources), client-side search (Pagefind), OAI-PMH and JSON-LD interfaces, sitemap, three syndication feeds (Atom, RSS 2.0, RDF) with legacy-path redirects, RelaxNG validation, contact + licence + Matomo + WCAG polish. Open: WCAG 2.2-AA audit on the live site, Matomo CI secrets, custom-domain decision, pre-publication preview decision (staging section of knowledge/pipeline.md). Current state and next entry point are in knowledge/journal.md.

Licence

Pipeline code, generated HTML output, and copied review images carry separate licences. Each is documented next to the artefact it covers; see CONTRIBUTING.md for the overview.