Skip to content

feat: DB-backed module settings with admin UI - #47

Merged
antosubash merged 34 commits into
mainfrom
feature/kind-wiles-92a9a3
Apr 21, 2026
Merged

feat: DB-backed module settings with admin UI#47
antosubash merged 34 commits into
mainfrom
feature/kind-wiles-92a9a3

Conversation

@antosubash

Copy link
Copy Markdown
Owner

Summary

Migrates the framework from SM_<MODULE>_* env vars to a DB-backed settings system with a live admin UI. Each module's BaseSettings class is hydrated from a single settings_override table at lifespan start, overrides are edited at /settings/modules, and changes hot-swap into app.state via a SettingsReloaded event without a restart (except where fields are explicitly requires_restart).

What changed

Framework plumbing

  • ModuleSettingsRegistry on SettingsServices — packages register their BaseSettings class via register_module_settings(app, cls).
  • SettingsStore — module-namespaced overrides; CRUD + typed get_overrides().
  • hydrate_settings(cls, store, package) — merges DB overrides onto defaults with typed coercion.
  • apply_changes_and_reload — validates via pydantic, persists the diff, hot-swaps app.state.<pkg>.settings, publishes SettingsReloaded.
  • Hydration runs at the start of the host lifespan before any on_startup hook.

Hosting split

  • BootstrapSettings (env-only, needed before DB opens) vs HostSettings (hydrated). Legacy Settings remains as a back-compat shim that inherits both.
  • Host registers itself as package="host" so the admin UI can edit host-level knobs.

Per-module migration

  • users, background_tasks, file_storage, datasets, settings — all now register their settings class. users.allow_signup gates at request time via a dependency so the toggle takes effect without a restart. Celery-owned fields are marked requires_restart.

REST API (modules/settings/settings/endpoints/module_api.py)

  • GET /api/settings/modules — list all registered modules + fields.
  • GET /api/settings/modules/{package} — single module.
  • PUT /api/settings/modules/{package} — apply changes (pydantic validation → 422 with per-field errors).
  • DELETE /api/settings/modules/{package}/{field} — reset one field to default.

Admin UI (/settings/modules)

  • ModulesEdit.tsx — sidebar + main panel with search filter.
  • ModuleForm.tsx — grouped fields, dirty tracking, per-field reset, 422 error display, "requires restart" badge when a restart-gated field is dirty.
  • FieldInput.tsx — type-driven rendering (bool→checkbox, int/float→number, json→textarea, string→text). Secret fields render masked with a "Set new value" reveal.

CLI

  • sm-settings import-from-env — one-shot migration: reads SM_<MODULE>_* env vars and writes them into settings_override.

Docs

  • Release note at docs/release-notes/2026-04-21-db-backed-settings.md.
  • .env.example trimmed to bootstrap-only (DB_URL, environment, secret, vite).
  • README config section rewritten.

Test plan

  • make test-py (853 passed)
  • make test-js (8 passed)
  • make lint (ruff, ty, biome, tsc, file-size all clean)
  • E2E toggle test (tests/e2e/test_settings_ui.py) verifies an admin can flip a setting and read it back through the REST API
  • Reviewer: spot-check /settings/modules in a running dev env — edit a non-restart field on users, confirm it takes effect on the next request without a restart
  • Reviewer: run sm-settings import-from-env against a populated .env and confirm the overrides land in settings_override

Notes for reviewers

  • Framework→plugin coupling rule (SM009) is preserved: _hydrate_step.py uses importlib.import_module("settings.hydrate") at call time rather than a top-level import.
  • docker-compose.yml deliberately keeps SM_BG_TASKS_* — those are deployment-plumbing for the Celery worker container, not runtime config.
  • The simplify pass (last commit) consolidated env-prefix resolution into settings.env_vars, removed dead code, added a no-op short-circuit to apply_changes_and_reload, and tightened the React form's dirty-tracking to avoid per-keystroke JSON.stringify bloat.

Replaces SM_<MODULE>_* env sprawl with DB-backed overrides behind the
existing pydantic BaseSettings schemas, plus a typed admin UI at
/settings/modules. Keeps only the four bootstrap env vars
(SM_DATABASE_URL, SM_ENVIRONMENT, SM_SECRET_KEY, SM_VITE_DEV_URL).
8-phase TDD plan covering registry/store/hydrator plumbing, hosting
bootstrap split, per-module migration, API, admin UI, import CLI,
cleanup, and final verification.
…uest time

Drops env_prefix/env_file from UsersSettings and wires register_settings
through register_module_settings so the module picks up DB-backed overrides
via the hosting lifespan's hydrate step. SM_ENVIRONMENT stays — it's a
host-level check, not a users field.

Since register_routes runs before app.state is reachable through the
per-request path, the fastapi-users register router is now always mounted
and gated by a new require_signup_enabled dependency that reads
request.app.state.users.settings.allow_signup. This preserves
hot-reloadability of allow_signup.

Test setup switches from monkeypatch.setenv("SM_USERS_*", ...) to seeding
the SettingsStore before lifespan entry. The production-guard tests now
pass secrets as kwargs instead of env vars.
- Use lowercase local vars for dynamically-imported classes (N806)
- Remove quoted type annotations (UP037)
- Use PEP 695 type parameter syntax in hydrate_settings (UP047)
- Remove unused noqa and TypeVar imports
Exposes a REST surface for the per-module settings admin UI over each
module's DB-backed BaseSettings:

- GET /api/settings/modules lists every registered module with hydrated
  field values plus metadata (type, requires_restart, group, is_secret).
- PUT /api/settings/modules/{package} validates with pydantic, persists
  overrides, hot-swaps app.state.<package>.settings, and publishes
  SettingsReloaded via apply_changes_and_reload. Secret fields whose
  value is the mask sentinel are silently dropped so echoing masked
  values never clobbers real secrets.
- DELETE /api/settings/modules/{package}/{field} clears a single override,
  re-hydrates, reassigns, and publishes SettingsReloaded.

collect_module_settings now folds in any registry-only packages (e.g.
"host") alongside plugin modules so the admin UI lists them too.
Walks app.state.settings.module_registry and, for every
SM_<PREFIX>_<FIELD> env var set, writes a SYSTEM-scoped override
via SettingsStore. Carries a small package -> env-prefix map
(background_tasks, file_storage) since Phase 3 stripped
env_prefix from pydantic configs.
… work

- Centralize env-prefix map in settings.env_vars; drop the dead
  _env_prefix_of that always returned "" and fix the UI env_var labels
- Publish SECRET_MASK and is_secret_field; stop importing private names
  from _module_settings into the endpoint module
- Collapse Settings shim via HostSettings + BootstrapSettings inheritance;
  delete the duplicated locale validator
- apply_changes_and_reload: skip persist + publish when the diff against
  current in-memory settings is empty; stop re-hydrating from DB when
  app.state already holds the validated instance
- ModuleForm: replace per-keystroke double JSON.stringify with Set of
  modified fields + useEffect to reset edit buffer when the module swaps
- FieldInput: merge int/float branches (same markup apart from step)
- Trim narrating comments
- Remove accidentally committed .superpowers/brainstorm/ state and
  gitignore the directory
Since Phase 3, UsersSettings no longer reads env — so SM_USERS_BOOTSTRAP_*
vars were silently ignored, the first-boot admin wasn't created, and
every E2E login-based test failed in CI.

Bootstrap fields are inherently one-shot seed inputs (not runtime DB
settings), so read them directly from os.environ in the two consumers
(bootstrap_admin_from_env and the dev-login hint). Settings values are
still preferred when tests set them explicitly.
Without wait_for_url, the goto('/settings/modules') races the login
POST's Set-Cookie response and sometimes lands on the login page instead
of the sidebar. Also swap the invalid lambda name= for a regex (Playwright
get_by_role accepts str or regex, not callables).
@antosubash
antosubash merged commit b1cc49c into main Apr 21, 2026
10 checks passed
antosubash added a commit that referenced this pull request Apr 21, 2026
Resolves three conflicts with main's db-backed-settings feature (PR #47)
and datasets view-routing fix (PR #48):

- modules/settings/pyproject.toml: kept both new blocks added on either
  side — [project.scripts] sm-settings entry (from main) AND [project.urls]
  metadata (from this branch).
- modules/settings/settings/pages/Modules.tsx: accepted main's deletion
  (replaced by ModulesEdit.tsx under the new admin UI). Our only edit
  here was the mechanical @simple-module → @simple-module-py scope
  rename, which no longer applies to a deleted file.
- modules/settings/settings/pages/ModulesEdit.tsx (new in main): updated
  its @simple-module/* imports to @simple-module-py/* to stay consistent
  with the npm scope rename on this branch.

Post-merge validators green:
  - scripts/check_metadata.py: All package metadata OK.
  - scripts/check_readmes.py: All READMEs OK.
  - scripts/bump_version.py 0.0.1 --check: All 17 packages at 0.0.1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant