Open-source A/B testing and feature flags that you run yourself.
Website: controlvar.com Β· Documentation: docs.controlvar.com
Watch the walkthrough videos on controlvar.com.
The core, the optional modules and the dashboard are Apache-2.0; the client SDKs under sdk/
are MIT. Controlvar is pre-1.0: see the
releases.
Controlvar was called Experimently before 0.35.0;
Upgrading to Controlvar says what
that changes for an install you already run.
A read-only copy runs at https://demo.controlvar.com: the dashboard as a viewer, with ShopLab and StreamPulse sending it simulated traffic. Creating and changing things is turned off there, and everything is rebuilt every night at 08:00 UTC. To create experiments, run it on your machine (below). The live demo says what is turned off there, and why.
You need Docker with Compose v2, curl and jq. Clone the repository:
git clone https://github.com/Controlvar/controlvar.git
cd controlvarStart the released images from deploy/compose/; nothing is built:
cd deploy/compose
./generate-env.sh
docker compose up -d --waitOpen http://localhost:8080 and sign in as admin@example.com with the
FIRST_SUPERUSER_PASSWORD that generate-env.sh wrote to deploy/compose/.env.
This stack has no demo data and no demo account. Its settings are in
deploy/compose/README.md.
Start the stack from the repository root:
docker compose up -d --waitCheck that you can sign in:
TOKEN=$(curl -s -X POST localhost:8000/api/v1/auth/login \
-H 'content-type: application/json' \
-d '{"email":"admin@demo.com","password":"Demo1234!"}' | jq -r .access_token)
curl -s localhost:8000/api/v1/auth/me -H "Authorization: Bearer $TOKEN" | jq .roleIt prints "ADMIN".
Open http://localhost:3000 and sign in with admin@demo.com / Demo1234!. The API is on http://localhost:8000, its docs at http://localhost:8000/api/v1/docs. Four containers come up β Postgres, Redis, the API and the dashboard β and the database is created and seeded on the way.
The first start builds the API and dashboard images from source.
docker compose down -v removes it all again, volumes included.
This stack is for your own machine. It runs with development settings and publishes its ports
on 127.0.0.1 only, and the demo account's password is on this page. To run Controlvar
where others can reach it, see Self-host it.
Quick start has the rest: the
service table, the verification commands, and the demo, tools and aws
profiles.
Two demo applications ship in the repo and drive the platform through the public SDK path, exactly as
a customer app would. ./demo/setup-local.sh starts everything.
- ShopLab (
demo/shoplab, http://localhost:3200): an e-commerce storefront running an A/B test, a multi-armed bandit, a multivariate test, a CUPED checkout test and a gradual-rollout flag. - StreamPulse (
demo/streampulse, http://localhost:3300): a phone drawn in a web page with the React SDK, with a device picker, targeting by OS version/region/tier, a kill switch, mutual exclusion and a scripted rollout story in which a crash spike triggers an automatic safety rollback.
Walkthrough: demo/DEMO_GUIDE.md. Step by step, with what you will see at each step: Guided examples.
- A/B Testing & Multivariate Experiments: Frequentist, sequential (mSPRT) and Bayesian analysis in the API; which of them the dashboard shows
- Sequential testing (beta): mSPRT with always-valid confidence intervals, so you can read results at any time. There are no O'Brien-Fleming or Pocock boundaries.
- Multiple-comparison correction: Benjamini-Hochberg by default, or Bonferroni, across each experiment's variants; stored with the experiment alongside its confidence level
- Sample Size Calculations: Automatic power analysis
- Results: events are stored in PostgreSQL and the API computes results from them
- Advanced Feature Flags: Targeting, gradual rollouts, and automated safety monitoring
- Enhanced Rules Engine: 20+ operators including semantic versioning, geo-distance, time windows
- RBAC: 4-tier role system (Admin, Developer, Analyst, Viewer)
- Audit Trail: creating, changing and deleting flags and experiments, and flag toggles, are recorded in every profile; the full profile adds warehouse changes, HMAC-SHA256 signing, SOC 2 / ISO 27001 reports and a JSON or CSV export for your compliance program
- Automated Safety: monitoring every 5 minutes by default, with automatic rollback capabilities
- Real-time Metrics: Error rate, latency, custom metrics tracking
- Auto-Rollback: Automatic rollback on threshold violations
- Configurable Thresholds: Per-feature flag safety configuration
- Rollback History: every rollback, automatic or manual, is stored with its reason; it also appears in the flag's audit history, unless that entry could not be written
- A flag serves two values, its on value and its off value; no rule or rollout splits users among three or more. For three or more experiences, run a multivariate experiment.
- A revenue, count, duration or custom metric's sequential, Bayesian, sample-size, breakdown and daily views are of the share of users who sent its event.
- Variant weights must add up to exactly 100%; there is no separate experiment-level traffic allocation.
- One deployment is one environment: run staging and production as separate deployments, each with its own API keys.
- Two-step sign-in can be required for chosen roles on Docker Compose, Helm and AWS CDK
installs; SAML sign-in is not available (OIDC is, with the
ssomodule). - Nothing deletes events, assignments or audit records by age, and no tool erases one person's
data. The platform's own analysis history is purged after
ANALYSIS_HISTORY_RETENTION_DAYS(90 days by default).
Limitations lists these and the rest, with what happens instead.
| Option | What you run | What CI runs |
|---|---|---|
| Docker Compose on one host | deploy/compose/compose.yml, on the published images |
starts it and checks the API answers |
| Helm on Kubernetes | charts/experimently |
installs it on a kind cluster |
| AWS CDK | infrastructure/cdk |
synthesises every stack; no CI job deploys it |
AWS CDK Deployment and Kubernetes have the steps.
There are 16 SDKs under sdk/, and 13 of them run assignment, flag evaluation and event
tracking against a live API in CI (the SDK Live Contract job of pr-qa-gate.yml).
| SDK | Install |
|---|---|
| JavaScript | npm install @controlvar/js-sdk |
| Python | pip install controlvar |
| Go | go get github.com/controlvar/controlvar/sdk/go |
| React | npm install @controlvar/react-sdk |
| React Native | npm install @controlvar/react-native-sdk |
| OpenFeature (Node) | npm install @controlvar/openfeature-provider @openfeature/server-sdk |
| OpenFeature (Python) | pip install controlvar-openfeature |
| Java | com.controlvar:controlvar-sdk (Maven Central) |
| .NET | dotnet add package Controlvar.SDK |
Every other SDK installs from a clone of this repository. The SDK guide says how, and what CI runs for each one.
The Controlvar MCP server is not an SDK. An AI assistant such as Claude Code starts it on your
computer to read your instance's feature flags, experiments and results with your API key, and it
changes no flag or experiment. The @controlvar/mcp-server package is on npm.
The MCP Server and the AI Endpoints has the command that adds it to Claude
Code, and the settings for Claude Desktop and Cursor. Its source is sdk/mcp.
- Documentation home
- Architecture overview
- Targeting rules: operators, attribute aliases and examples
- The Controlvar books: engineering, programming and the craft of shipping software, taught through this codebase, for newcomers to a language or framework, engineers who already code, and contributors (CC BY 4.0)
- Controls that support your compliance program: append-only audit log, role-based access,
per-flag safety monitoring with automatic rollback; the
complianceandhipaamodules add tamper-evident audit signing, compliance report packs and PHI encryption. On 2026-10-13 the project held no SOC 2, ISO 27001 or HIPAA attestation, and it claims none. - Passwords and API keys: stored hashed; an API key is shown once, when it is created
- API keys: any signed-in user can create one for themselves, and a key acts as its creator on the tracking and flag-evaluation routes, so a VIEWER's key can send assignments and events
- Encryption: AES-256 at rest (KMS) and HTTPS in transit at the load balancer when deployed with the provided CDK
- Network Security: VPC isolation, security groups, and an HTTPS-only load balancer (HTTP redirects to HTTPS) in front of the API and the dashboard (CDK deployment). The CDK does not deploy a WAF or CloudFront
One codebase, two profiles. The core profile is backend/ and frontend/: experiments and
feature flags end to end, targeting with 20+ operators, gradual rollouts, safety monitoring with
automatic rollback, scheduling, the full statistics (frequentist and Bayesian, sequential testing,
CUPED, multi-armed bandits, mutual exclusion groups and global holdouts, dimensional
breakdowns, experiment overlap and interaction detection (beta),
live results), the four built-in roles, audit logging, API keys, alerting
and every SDK. The full profile adds the optional modules under modules/, which plug into
the core through the registration hooks in backend/app/core/:
| Module | What it adds |
|---|---|
workspaces |
Group users into teams, with workspace roles and email invites. Grouping only: workspace roles play no part in access to experiments and flags |
rbac |
Roles beyond the built-in four, and permissions granted directly to a user, optionally until a date. Both add to what the user's built-in role allows |
sso |
OIDC identity providers with just-in-time provisioning and role mapping. SAML 2.0 sign-in is not available yet; use OIDC |
hipaa |
PHI encryption, PHI access records stamped with a six-year retention (nothing deletes them), BAA records |
compliance |
SOC 2 / ISO 27001 reports, signed audit exports |
warehouse |
Warehouse analysis (beta): run an experiment's analysis on tables in your own warehouse, under /api/v1/warehouse/analysis (connections, sources, runs). Snowflake and BigQuery are available; GET /api/v1/warehouse/analysis/connectors lists which connectors are. See docs/api/warehouse-analytics.md |
integrations |
Jira, Salesforce and GitHub |
counters |
Routes to increment and read assignment and conversion counters in DynamoDB; assignment and tracking do not write them |
etl |
Glue crawlers, Athena partitions and scheduled jobs |
split_url |
Split URL experiments: each variant is its own URL. Your server or edge asks the API where to send each visitor (on AWS an optional CloudFront router can). The AWS installer installs the core profile, so it does not include them |
GET /api/v1/modules reports the running profile and the installed modules β ask it first, rather
than inferring the profile from a status code. A core deployment is the repository with modules/
deleted (make core-build proves it builds, boots and passes its tests that way), so a module's
routes are not mounted and its URLs 404 like any other unknown path. Three URLs are declared by
a core router but implemented by a module β /api/v1/compliance/reports/{standard},
/api/v1/compliance/export and /api/v1/experiments/{id}/split-url/preview β and those answer
501, as does creating an experiment with experiment_type=split_url: the route exists in both
profiles, so it says "not installed" rather than leaving you to wonder about a typo.
The two profiles are under the same licence. Details in
docs/getting-started/modules.md.
| Part of the repository | Licence |
|---|---|
Everything except sdk/ β the core, the modules, the dashboard, the docs |
Apache-2.0 |
sdk/ (all client SDKs and OpenFeature providers) |
MIT |
Third-party dependency licences are listed in THIRD_PARTY_LICENSES.md; attribution is in NOTICE. Security reports go to SECURITY.md, not to public issues. Contributions are accepted under the DCO β see CONTRIBUTING.md.
CONTRIBUTING.md covers the development setup, the test suites and the core/modules boundary.
ROADMAP.md says what is planned Now, Next and Later, and how to influence it.
