Skip to content

Commit 8ffdcc7

Browse files
Lucais11rubyskcmartin
authored
(WIP)Add multi-container machines doc (#2083)
* (WIP)Add multi-container machines doc * Clean up indentation * replace raw_value with base64 encoded value * Update machines/guides-examples/multi-container-machines.html.markerb Co-authored-by: Kristin Martin <[email protected]> * Update machines/guides-examples/multi-container-machines.html.markerb Co-authored-by: Kristin Martin <[email protected]> * Update machines/guides-examples/multi-container-machines.html.markerb Co-authored-by: Kristin Martin <[email protected]> * Update machines/guides-examples/multi-container-machines.html.markerb Co-authored-by: Kristin Martin <[email protected]> * Update machines/guides-examples/multi-container-machines.html.markerb Co-authored-by: Kristin Martin <[email protected]> * Wording change --------- Co-authored-by: Sam Ruby <[email protected]> Co-authored-by: Kristin Martin <[email protected]>
1 parent ed7daa9 commit 8ffdcc7

File tree

2 files changed

+342
-0
lines changed

2 files changed

+342
-0
lines changed

‎machines/guides-examples/multi-container-machines.html.markerb

Lines changed: 341 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,341 @@
1+
---
2+
title: Multi-container Machines
3+
layout: docs
4+
order: 28
5+
nav: machines
6+
---
7+
8+
Fly Machines support running multiple containers per virtual machine using the `containers` array. This feature is currently available through the Machines API and requires using Pilot as the init system. This capability is useful for co-locating services such as your application server, log shippers, metrics collectors, or sidecars.
9+
10+
## Common Sidecar Use Cases
11+
12+
Sidecars are a powerful way to co-locate supporting services alongside your application. This pattern is especially helpful for apps built with multi-tenant or per-user architectures, such as AI agents, dev environments, or SaaS dashboards. Some practical uses include:
13+
14+
- **Metrics and Logs:** Run a Prometheus exporter or a log shipper like Vector to forward application logs or metrics without bundling that logic into your app.
15+
- **Secrets Agents:** Use a sidecar to fetch secrets from a provider like Vault or cloud metadata services. This keeps your app container lightweight and secure.
16+
- **Load Smoothing:** Use nginx or Envoy as a reverse proxy for rate limiting, retries, or graceful error handling. This is great for handling bursts in traffic or external API instability.
17+
- **Storage or Sync Layers:** Mount and manage local state using sidecars like LiteFS or file sync daemons. You can keep persistent logic separated from your app's logic.
18+
- **AI Agents or Workers:** For agent-based or queued workloads, you can spin up workers or schedulers in separate containers and coordinate them using `depends_on`.
19+
20+
These patterns mirror what we've seen from customers building production systems on Fly Machines with Pilot and the `containers` array. Each container runs in its own isolated process tree, and Pilot ensures the startup order and health status are respected. While containers share the same kernel and VM, they are isolated at the process and filesystem level.
21+
22+
## Defining Containers
23+
24+
When creating a Machine via the API, you can specify a `containers` array, each containing a [`ContainerConfig`](https://docs.machines.dev/#model/flycontainerconfig). Each container can have its own image, environment variables, health checks, startup commands, attached files, and dependencies.
25+
26+
Flyd (our [in-house orchestrator](https://fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/)), handles pulling images and making them accessible to Pilot, the init system responsible for running the containers and managing their lifecycle. Pilot builds a dependency graph using container startup conditions and runs containers accordingly.
27+
28+
## Example Configuration
29+
30+
```json
31+
{
32+
"containers": [
33+
{
34+
"image": "registry.fly.io/my-app:latest",
35+
"env": {
36+
"PORT": "8080"
37+
},
38+
"services": [
39+
{
40+
"internal_port": 8080,
41+
"protocol": "tcp",
42+
"ports": [
43+
{
44+
"port": 80,
45+
"handlers": ["http"]
46+
}
47+
]
48+
}
49+
],
50+
"health_checks": [
51+
{
52+
"type": "http",
53+
"path": "/health",
54+
"interval": "10s",
55+
"timeout": "5s",
56+
"grace_period": "5s",
57+
"success_threshold": 2,
58+
"failure_threshold": 3
59+
}
60+
]
61+
},
62+
{
63+
"image": "registry.fly.io/log-shipper:latest",
64+
"env": {
65+
"LOG_LEVEL": "info"
66+
},
67+
"depends_on": [
68+
{
69+
"container": "my-app",
70+
"condition": "healthy"
71+
}
72+
]
73+
}
74+
]
75+
}
76+
```
77+
78+
In this configuration, the `log-shipper` container depends on the `my-app` container being healthy before starting.
79+
80+
## Container Dependencies
81+
82+
You can define dependencies between containers using the `depends_on` field. Supported conditions include:
83+
84+
```json
85+
"depends_on": [
86+
{
87+
"container": "sidecar1",
88+
"condition": "healthy"
89+
}
90+
]
91+
```
92+
93+
Valid values for `condition`:
94+
95+
- `healthy`: Container will wait until the dependency container passes its health checks.
96+
- `started`: Container will start as soon as the dependency container has started.
97+
- `exited_successfully`: Container will wait until the dependency container has exited with a success code.
98+
99+
Internally, Pilot models these dependencies as a directed graph and walks it to orchestrate the correct container startup order.
100+
101+
## Health Checks
102+
103+
Each container specifies its own health checks. The system supports different types of health checks:
104+
105+
1. **TCP health checks**: Checks if a port is accepting connections
106+
107+
```json
108+
"tcp": {
109+
"port": 6379
110+
}
111+
```
112+
113+
1. **HTTP health checks**: Makes HTTP requests to an endpoint
114+
115+
```json
116+
"http": {
117+
"port": 80,
118+
"method": "GET",
119+
"path": "/",
120+
"scheme": "http"
121+
}
122+
```
123+
124+
1. **Exec health checks**: Runs a command inside the container
125+
126+
```json
127+
"exec": {
128+
"command": ["sh", "-c", "pg_isready -U postgres"]
129+
}
130+
```
131+
132+
### Health Check Configuration Options
133+
134+
- `name`: Unique identifier for the health check
135+
- `interval`: How often to run the check (e.g., "10s")
136+
- `grace_period`: Initial period after container starts before checks begin
137+
- `timeout`: Maximum time to wait for a check to complete
138+
- `success_threshold`: Number of consecutive successful checks required to change status to healthy
139+
- `failure_threshold`: Number of consecutive failed checks before marking as unhealthy
140+
141+
## Security Considerations
142+
143+
Containers in a Machine share the same kernel and VM, but are isolated at the process and filesystem level. Failures in one container won't directly crash others, but they don't provide the same level of isolation as across VMs.
144+
145+
## Deploying Multi-container Machines
146+
147+
There are several ways to deploy multi-container machines on Fly.io. Choose the method that best fits your workflow:
148+
149+
### Using flyctl
150+
151+
You can define your Machine configuration in a JSON file (e.g., `machine-config.json`) and create the Machine using the Fly CLI:
152+
153+
```bash
154+
flyctl machines create -f machine-config.json
155+
```
156+
157+
You can also launch a multi-container app interactively using `flyctl machine run`:
158+
159+
```bash
160+
flyctl machine run --machine-config cli-config.json \
161+
--autostart=true --autostop=suspend \
162+
--port 80:8080/tcp:http --port 443:8080/tcp:http:tls \
163+
--vm-cpu-kind shared --vm-cpus 1 --vm-memory 256
164+
```
165+
166+
Alternatively, you can make a direct API call using `curl`:
167+
168+
```bash
169+
curl -X POST "https://api.fly.io/v1/apps/your-app-name/machines" \
170+
-H "Authorization: Bearer your-api-token" \
171+
-H "Content-Type: application/json" \
172+
-d @machine-config.json
173+
```
174+
175+
Refer to the [Machines API documentation](https://docs.machines.dev/#tag/machines) for more details.
176+
177+
### Using fly launch & fly deploy
178+
179+
Starting with `flyctl v0.3.113`, you can run multiple containers using `fly deploy`. This example demonstrates using a nginx container as a rate limiter and a custom app container.
180+
181+
#### 1. Create cli-config.json
182+
183+
```json
184+
{
185+
"config": {
186+
"containers": [
187+
{
188+
"name": "nginx",
189+
"image": "nginx:latest",
190+
"files": [
191+
{
192+
"guest_path": "/etc/nginx/conf.d/default.conf",
193+
"local_path": "nginx.conf"
194+
}
195+
],
196+
"depends_on": [
197+
{
198+
"container": "echo",
199+
"condition": "healthy"
200+
}
201+
]
202+
},
203+
{
204+
"name": "echo",
205+
"image": "ealen/echo-server",
206+
"health_checks": [
207+
{
208+
"type": "exec",
209+
"command": ["wget", "-q", "--spider", "http://localhost"]
210+
}
211+
]
212+
}
213+
]
214+
}
215+
}
216+
```
217+
218+
#### 2. Update fly.toml
219+
220+
```
221+
[experimental]
222+
machine_config = 'cli-config.json'
223+
container = 'echo'
224+
225+
[http_service]
226+
internal_port = 8080
227+
```
228+
229+
The container value determines which container will be replaced by your app image built by `fly deploy`.
230+
231+
#### 3. Run fly deploy
232+
233+
```bash
234+
fly deploy
235+
```
236+
237+
This builds and replaces the `echo` container, runs both containers on the same machine, and configures traffic through nginx.
238+
239+
You can also inline the `machine_config` in your `fly.toml` using triple quotes. The first character inside the string must be `{`:
240+
241+
```
242+
machine_config = '''{
243+
"containers": [
244+
…
245+
]
246+
}'''
247+
```
248+
249+
## Example Configurations
250+
251+
Here are some practical examples of multi-container setups you can use as a starting point:
252+
253+
### Rate Limiter Sidecar
254+
255+
To mitigate excessive traffic, you can run an nginx container as a sidecar to limit requests per second using the `ngx_http_limit_req_module`. Below is a simplified container configuration that enables rate limiting:
256+
257+
```json
258+
{
259+
"config": {
260+
"containers": [
261+
{
262+
"name": "nginx",
263+
"image": "nginx:latest",
264+
"files": [
265+
{
266+
"guest_path": "/etc/nginx/conf.d/default.conf",
267+
"raw_value": "bGltaXRfcmVxX3pvbmUgJGJpbmFyeV9yZW1vdGVfYWRkciB6b25lPW15X3pvbmU6MTBtIHJhdGU9MXIvczsKCnNlcnZlciB7CiAgbGlzdGVuIDgwODA7CgogIGxvY2F0aW9uIC8gewogICAgbGltaXRfcmVxIHpvbmU9bXlfem9uZTsKICAgIHByb3h5X3Bhc3MgaHR0cDovL2xvY2FsaG9zdDo4MDsKICB9Cn0="
268+
}
269+
]
270+
},
271+
{
272+
"name": "echo",
273+
"image": "ealen/echo-server"
274+
}
275+
]
276+
}
277+
}
278+
```
279+
280+
When attaching files to containers, the `raw_value` field must contain base64-encoded content. The nginx configuration shown in the JSON above corresponds to this configuration:
281+
282+
```
283+
limit_req_zone $binary_remote_addr zone=my_zone:10m rate=1r/s;
284+
285+
server {
286+
listen 8080;
287+
288+
location / {
289+
limit_req zone=my_zone;
290+
proxy_pass http://localhost:80;
291+
}
292+
}
293+
```
294+
295+
When the container starts, Fly will decode this value and write the original configuration to the specified `guest_path`.
296+
297+
To complete this configuration, you should also include:
298+
299+
- A `services` block that exposes internal port 8080 for Fly Proxy
300+
- A `guest` block specifying resources like CPU and memory
301+
- Optionally, `depends_on` fields or health checks if you want nginx to wait for the app
302+
303+
### LiteFS Sidecar with Health Checks
304+
305+
You can also run LiteFS as a sidecar with a health check configured to ensure it's operating correctly:
306+
307+
```json
308+
{
309+
"config": {
310+
"containers": [
311+
{
312+
"name": "litefs",
313+
"image": "flyio/litefs:latest",
314+
"env": {
315+
"LITEFS_DIR": "/data",
316+
"LITEFS_CONFIG": "/etc/litefs.yml"
317+
},
318+
"health_checks": [
319+
{
320+
"type": "http",
321+
"port": 8080,
322+
"path": "/info",
323+
"method": "GET",
324+
"scheme": "http",
325+
"grace_period": 500,
326+
"interval": 1000,
327+
"timeout": 1000,
328+
"delay_start": 0,
329+
"success_threshold": 1,
330+
"failure_threshold": 5
331+
}
332+
]
333+
}
334+
]
335+
}
336+
}
337+
```
338+
339+
<div class="important icon">
340+
For more information about container configuration options, refer to the [Machines API documentation](https://docs.machines.dev/#model/flycontainerconfig).
341+
</div>

‎partials/_machines_nav.html.erb

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
links: [
3838
{ text: "Machine restart policy", path: "/docs/machines/guides-examples/machine-restart-policy/" },
3939
{ text: "Machine sizing", path: "/docs/machines/guides-examples/machine-sizing/" },
40+
{ text: "Multi-container Machines", path: "/docs/machines/guides-examples/multi-container-machines/" },
4041
{ text: "Network policies", path: "/docs/machines/guides-examples/network-policies/" },
4142
{ text: "Run user code on Fly Machines", path: "/docs/machines/guides-examples/functions-with-machines/" },
4243
{ text: "One app per customer - why?", path: "/docs/machines/guides-examples/one-app-per-user-why/" }

0 commit comments

Comments
 (0)