Skip to content

Add support for container restart policy - #1258

Open
JaewonHur wants to merge 32 commits into
apple:mainfrom
JaewonHur:container-restart
Open

JaewonHur wants to merge 32 commits into
apple:mainfrom
JaewonHur:container-restart

Conversation

@JaewonHur

@JaewonHur JaewonHur commented Feb 24, 2026 •

Copy link
Copy Markdown
Contributor

Closes #286.

The first milestone is the restart policies no, on-failure[:<max-retries>] and always, which behave like the Docker restart policies. The CLI exposes them as --restart. The design can be extended later to support auto-start (#158, #1201).

CLI

--restart on container run and container create:

┌────────────────┬───────────────────────────────────────────────────────────────────────────────────────────┐
│     Value      │                                         Behavior                                          │
├────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ no (default)   │ Never restart.                                                                            │
├────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ on-failure     │ Restart on a non-zero exit code, with unlimited retries.                                  │
├────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ on-failure:<n> │ Same, but give up after n automatic restarts. on-failure:0 means unlimited, as in         │
│                │ Docker/Moby (MaximumRetryCount == 0).                                                     │
├────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ always         │ Restart on any exit, unless the container was stopped manually.                           │
└────────────────┴───────────────────────────────────────────────────────────────────────────────────────────┘
  • The spelling is Docker-compatible. Swift spellings such as onFailure are rejected, as are malformed counts (on-failure:-1, on-failure:abc, on-failure:) and counts with other policies (always:3).
  • --rm cannot be combined with any restart policy other than no. This is rejected in the CLI flag validation and again in ContainersService.create, before the container is created.
  • unless-stopped is deliberately left out as we need an additional persistent state for remembering whether a container was manually stopped or not. Will figure out after this.

Persistent and runtime state

ContainerCreateOptions (persistent)

  • restartPolicy: RestartPolicy remembers the policy across the container life cycle. It defaults to no when decoding options that have no restartPolicy.
  • RestartPolicy is .no, .always or .onFailure(maximumRetryCount:). The retry limit is part of the policy, not a separate option.

ContainerState (runtime)

  • StoppedState(startError: Error?, exitStatus: ExitStatus?) decides the restart logic when a container exits.
  • manualStopped: Bool records that the user stopped the container. stop and kill set it, and a new start (by the user or the scheduler) clears it. It is not persisted, so it is false after loadAtBoot.
  • restartCount: Int counts automatic restarts, for on-failure:. The scheduler increments it on each restart. A start that doesn't come from the scheduler, such as container start, resets it, so the user gets a fresh retry budget. It is not reset by a stable run, only the backoff is.
  • backOff: Duration? is the restart backoff timer. It is nil by default. On the first non-manual exit that qualifies for a restart it is set to 100ms, then doubled on each further exit up to 10s. It is reset to nil on a start error, a manual stop, or a policy that doesn't restart.

A new restarting status is added to RuntimeStatus. A container that is going to restart enters restarting, waits for its backOff, then runs bootstrap followed by startProcess. bootstrapped is also added as an explicit status between stopped and running. Stopping or killing a container while it is restarting cancels the pending restart.

Restart logic (handleContainerExit)

  • no: stay stopped.
  • on-failure[:n]: stay stopped on a start error, a manual stop, a zero exit code, or when restartCount has reached n (if n > 0). Otherwise restart.
  • always: stay stopped on a start error or a manual stop. Otherwise restart.

A restart scheduler task in ContainersService receives the ids of stopped containers through an async queue and restarts those that match the policy. The scheduler doesn't hold the lock. Every state-changing operation it performs, such as bootstrap, is lock-protected, so this adds no new race.

Stability call

A container that stays running for 10 seconds is marked stable, and its backOff is reset. A monitor (similar to ExitMonitor) registers the callback on a successful startProcess.

@github-actions github-actions Bot added the facet/cli Work related to generic or cross-cutting CLI features label Feb 24, 2026
@saehejkang

Copy link
Copy Markdown
Contributor

@JaewonHur Busy week and I hope to get to review this here soon. I saw your comment and I am all for following the direction we take in this PR. Though it might be worthwhile to get started on the discussion on how we want to handle test cases and maybe work together on that piece?

@JaewonHur

Copy link
Copy Markdown
Contributor Author

Hi @saehejkang, sorry for moving forward on my own.

I just did minimal implementation today, but I'll definitely put our name together here.
This PR now just contains prototype implementation, and anything can be changed! Please take a look :)

Further things to do might be:

  1. supporting --restart systemStart (or make always to start on system start?)
  2. Add restart counter for onFailure.
  3. Also, testing for restarting container across system can be added.

@saehejkang saehejkang left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went over things as best as I could for now. Lots of changes that I have not gotten a chance to dig deeper into (the scheduler). Don't worry about taking over and wrapping this up!

case stopped
/// The object is waiting to be restarted.
case restarting
/// The object is currently bootstrapped.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does the bootstrapped status mean? The container or sandbox is stopped but ready to be started?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bootstrapped means (by itself) the container is bootstrapped (i.e., bootstrap function has been successfully executed on that container).

Before this update, the state was same stopped whether it is bootstrapped or not. The only difference was SandboxClient is allocated or not---I personally felt that somewhat weird, but left there.

But after adding restarting state, it becomes more weird, since container state should go restarting (waiting for backOff), then stopped (after bootstrap) then running`.

BTW, now we split the RuntimeStatus to ContainerStatus and SandboxStatus, so it only applies to containers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While writing this comment, I found that I missed setting the container restarting in scheduler lol.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bootstrapped means (by itself) the container is bootstrapped (i.e., bootstrap function has been successfully executed on that container).

I understand that this has to do with the SandboxService and SandboxClient. Except, I have yet to understand what is exactly the point of those two things?

BTW, now we split the RuntimeStatus to ContainerStatus and SandboxStatus, so it only applies to containers.

So it seems like Containers and the Sandbox are two separate entities with different capabilities. Goes back to my question above and how they differentiate.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Container and Sandbox are separate entities with different lifecycle (though they are tightly coupled).

Sandbox is more for managing the lifecycle of VM.

Sandbox is only alive after the Container successfully executed bootstrap.

return results
}

public func runRestartScheduler() throws {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never thought I would see a scheduler again after OS in college 😆.

@saehejkang

Copy link
Copy Markdown
Contributor

supporting --restart systemStart (or make always to start on system start?)

Solid foundation work was done with the restart policy and I am sure it could follow the same pattern?

Add restart counter for onFailure.

What would be the use case for this? Could it be used to automatically stop trying to restart a container after a certain count?

Also, testing for restarting container across system can be added.

Hopefully we can use something like #1176 to be a sort of final e2e test of all the policies.

@JaewonHur

Copy link
Copy Markdown
Contributor Author

Solid foundation work was done with the restart policy and I am sure it could follow the same pattern?

Thanks, if you make PR to my branch, let me merge it :)

What would be the use case for this? Could it be used to automatically stop trying to restart a container after a certain count?

Yes. Docker supports that functionality (https://docs.docker.com/engine/containers/start-containers-automatically/)

@saehejkang

Copy link
Copy Markdown
Contributor

@JaewonHur I have not forgotten about this PR. Just another busy week and I hope to come back to this and #1201 in the near future.

@saehejkang

saehejkang commented Apr 7, 2026 •

Copy link
Copy Markdown
Contributor

@JaewonHur Back at it and here to help wrap this up! I know you are very busy so feel free to assign me to anything from the implementation steps. I think it is best to wrap up restart before I work on completing auto-start.

chrisgeo added a commit to full-chaos/container that referenced this pull request May 15, 2026
While this PR was sitting in draft, apple#1258 (by JaewonHur,
reviewed by saehejkang) appeared with a substantial in-flight restart
manager implementation. To avoid a future merge conflict on every field
of this contract, the fork's SDK shape is reshaped to match apple#1258
verbatim before any reviewer sees this PR.

Changes
-------

RestartPolicy: struct-with-mode-and-maxRetries -> bare String-backed
enum { no, onFailure, always }. Matches apple#1258 line-for-line. Drops the
'unless-stopped' mode and the 'maxRetries' field; both are intentionally
deferred so this PR mirrors upstream's deliberately conservative initial
scope. Follow-ups will land them after apple#1258 merges (CHAOS-1321c).

ContainerCreateOptions: restartPolicy goes from optional to
non-optional, defaulting to .no. Custom 'init(from:)' uses
'decodeIfPresent ?? .no' so older 'options.json' blobs (no field) still
decode cleanly. This is the forward-compat guarantee the original PR
description promised but did not actually implement.

Flags.swift: '--restart' goes from 'String?' + a hand-rolled
'parseRestartPolicy' helper to '@option var restart: RestartPolicy =
.no'. ArgumentParser handles validation natively via a new
'extension RestartPolicy: ExpressibleByArgument {}'; the parser is
deleted entirely. As a side benefit, invalid input now produces
ArgumentParser's standard 'Invalid value for --restart' error instead
of being silently dropped.

ContainerRun.swift / ContainerCreate.swift: thread 'managementFlags
.restart' through directly. ContainerCreate was missing the wiring
entirely in the original PR (the flag declared but never read on
the create path) -- fixed here.

Tests/ContainerResourceTests/RestartPolicyTests.swift (new): 8 tests
covering bare-string encode/decode, every-case round-trip, unknown-
string rejection, and the legacy 'options.json' forward-compat
invariant (legacy blob without restartPolicy decodes with
restartPolicy == .no).

Verification
------------

  swift build  -> Build complete! (21.99s), exit 0.
  swift test --filter RestartPolicyTests
               -> 8 tests passed in 0.001s.
  LSP diagnostics -> clean across all 6 files.

Upstream alignment matrix (this PR vs apple#1258)
-----------------------------------------------------------

  RestartPolicy shape           bare enum         match
  unless-stopped mode           absent            match
  on-failure:N retries          absent            match
  restartPolicy optionality     non-optional      match
  decodeIfPresent default       .no               match
  Flag binding                  ExpressibleByArg  match
  RuntimeStatus -> ContainerStatus rename
                                NOT in scope      defer

The RuntimeStatus rename and the .restarting/.bootstrapped state
additions stay deferred -- they belong to the daemon-side enforcement
work, which is what apple#1258 actually does. This PR remains data-shape
only on the fork side; enforcement arrives when apple#1258 lands upstream
and the fork bumps its sync.

Refs CHAOS-1321, CHAOS-1385.
Sponsors apple#1258, apple#286.
chrisgeo added a commit to full-chaos/container that referenced this pull request May 24, 2026
While this PR was sitting in draft, apple#1258 (by JaewonHur,
reviewed by saehejkang) appeared with a substantial in-flight restart
manager implementation. To avoid a future merge conflict on every field
of this contract, the fork's SDK shape is reshaped to match apple#1258
verbatim before any reviewer sees this PR.

Changes
-------

RestartPolicy: struct-with-mode-and-maxRetries -> bare String-backed
enum { no, onFailure, always }. Matches apple#1258 line-for-line. Drops the
'unless-stopped' mode and the 'maxRetries' field; both are intentionally
deferred so this PR mirrors upstream's deliberately conservative initial
scope. Follow-ups will land them after apple#1258 merges (CHAOS-1321c).

ContainerCreateOptions: restartPolicy goes from optional to
non-optional, defaulting to .no. Custom 'init(from:)' uses
'decodeIfPresent ?? .no' so older 'options.json' blobs (no field) still
decode cleanly. This is the forward-compat guarantee the original PR
description promised but did not actually implement.

Flags.swift: '--restart' goes from 'String?' + a hand-rolled
'parseRestartPolicy' helper to '@option var restart: RestartPolicy =
.no'. ArgumentParser handles validation natively via a new
'extension RestartPolicy: ExpressibleByArgument {}'; the parser is
deleted entirely. As a side benefit, invalid input now produces
ArgumentParser's standard 'Invalid value for --restart' error instead
of being silently dropped.

ContainerRun.swift / ContainerCreate.swift: thread 'managementFlags
.restart' through directly. ContainerCreate was missing the wiring
entirely in the original PR (the flag declared but never read on
the create path) -- fixed here.

Tests/ContainerResourceTests/RestartPolicyTests.swift (new): 8 tests
covering bare-string encode/decode, every-case round-trip, unknown-
string rejection, and the legacy 'options.json' forward-compat
invariant (legacy blob without restartPolicy decodes with
restartPolicy == .no).

Verification
------------

  swift build  -> Build complete! (21.99s), exit 0.
  swift test --filter RestartPolicyTests
               -> 8 tests passed in 0.001s.
  LSP diagnostics -> clean across all 6 files.

Upstream alignment matrix (this PR vs apple#1258)
-----------------------------------------------------------

  RestartPolicy shape           bare enum         match
  unless-stopped mode           absent            match
  on-failure:N retries          absent            match
  restartPolicy optionality     non-optional      match
  decodeIfPresent default       .no               match
  Flag binding                  ExpressibleByArg  match
  RuntimeStatus -> ContainerStatus rename
                                NOT in scope      defer

The RuntimeStatus rename and the .restarting/.bootstrapped state
additions stay deferred -- they belong to the daemon-side enforcement
work, which is what apple#1258 actually does. This PR remains data-shape
only on the fork side; enforcement arrives when apple#1258 lands upstream
and the fork bumps its sync.

Refs CHAOS-1321, CHAOS-1385.
Sponsors apple#1258, apple#286.
@tonycoco

Copy link
Copy Markdown

This is also quite nice when NFS/CIFS/SMB mounts fail and a container needs to restart for connection purposes.

@stephenlclarke

Copy link
Copy Markdown
Contributor

I reviewed this again from the downstream stephenlclarke/container-compose side alongside apple/container#286. I do not want to open a competing restart-policy PR while this one is active.

The latest fork work is now split into narrow, PR-shaped branches rather than the older integration commits:

  • stephenlclarke/container branch restart-policy-create-options, commit c5668c19d139b1aeb7e2529cb1dedd01fb4532c1: create-time restart policy model, ContainerCreateOptions.restartPolicy, Docker-compatible container run/create --restart, and --rm conflict validation.
  • stephenlclarke/container branch restart-policy-runtime, commit b41bb830db708bc839c94e01c8a75c7fecbe3db0: runtime restart scheduling, manual-stop suppression, cancellation, retry limits, backoff, and stable-run reset.
  • stephenlclarke/container branch restart-policy-timing, commit 8b1eff72481fa497328414e0483a08c768826f1a: optional retry delay and successful-run window controls.
  • stephenlclarke/container-compose branch compose-restart-policy-mapping, commit 06cc3220c03b8ce0ab2eaf83ed81c08aca7f74b4: downstream service restart and Deploy restart_policy mapping, plus a local-only Docker Compose V2 parity script that is intentionally not wired into CI.

I think the clean review stack is:

  1. Create-time API and CLI contract only.

    This establishes the public shape without restarting containers yet. It covers:

    • no
    • always
    • unless-stopped
    • on-failure
    • on-failure:<max-retries>

    It also keeps old serialized create options decoding as the default no policy and rejects --rm with any non-default restart policy before container creation.

  2. Runtime restart behavior.

    This then applies the stored policy when the init process exits. It covers manual-stop suppression, on-failure retry limits, always / unless-stopped, backoff, stable-run reset, and cancellation when a container is stopped, killed, deleted, or manually restarted.

  3. Timing controls.

    This follows after the base behavior is accepted and adds optional retry delay / successful-run window fields. That lets downstream tools map Compose Deploy restart_policy.delay and restart_policy.window without putting Compose-specific behavior into apple/container.

Two details I would keep explicit in the first slice:

  • the CLI syntax should stay Docker-compatible (on-failure, not Swift enum spelling such as onFailure);
  • on-failure:0 should mean unlimited retries, not zero retries. The fork now documents this in code as Docker/Moby behavior; Colima delegates this to Docker Engine, and Moby treats MaximumRetryCount == 0 as unlimited.

For stephenlclarke/container-compose, the direct upstream dependency is just the generic apple/container primitive: a stored restart policy, then runtime behavior, then optional timing. Compose precedence rules, one-off compose run behavior, Deploy-model normalization, and local Docker Compose parity checks stay downstream.

Would Apple prefer this PR branch to be refreshed into that smaller stacked shape, or would a new first PR for only the create-time API/CLI contract be easier to review?

chrisgeo added a commit to full-chaos/container that referenced this pull request Aug 28, 2026
While this PR was sitting in draft, apple#1258 (by JaewonHur,
reviewed by saehejkang) appeared with a substantial in-flight restart
manager implementation. To avoid a future merge conflict on every field
of this contract, the fork's SDK shape is reshaped to match apple#1258
verbatim before any reviewer sees this PR.

Changes
-------

RestartPolicy: struct-with-mode-and-maxRetries -> bare String-backed
enum { no, onFailure, always }. Matches apple#1258 line-for-line. Drops the
'unless-stopped' mode and the 'maxRetries' field; both are intentionally
deferred so this PR mirrors upstream's deliberately conservative initial
scope. Follow-ups will land them after apple#1258 merges (CHAOS-1321c).

ContainerCreateOptions: restartPolicy goes from optional to
non-optional, defaulting to .no. Custom 'init(from:)' uses
'decodeIfPresent ?? .no' so older 'options.json' blobs (no field) still
decode cleanly. This is the forward-compat guarantee the original PR
description promised but did not actually implement.

Flags.swift: '--restart' goes from 'String?' + a hand-rolled
'parseRestartPolicy' helper to '@option var restart: RestartPolicy =
.no'. ArgumentParser handles validation natively via a new
'extension RestartPolicy: ExpressibleByArgument {}'; the parser is
deleted entirely. As a side benefit, invalid input now produces
ArgumentParser's standard 'Invalid value for --restart' error instead
of being silently dropped.

ContainerRun.swift / ContainerCreate.swift: thread 'managementFlags
.restart' through directly. ContainerCreate was missing the wiring
entirely in the original PR (the flag declared but never read on
the create path) -- fixed here.

Tests/ContainerResourceTests/RestartPolicyTests.swift (new): 8 tests
covering bare-string encode/decode, every-case round-trip, unknown-
string rejection, and the legacy 'options.json' forward-compat
invariant (legacy blob without restartPolicy decodes with
restartPolicy == .no).

Verification
------------

  swift build  -> Build complete! (21.99s), exit 0.
  swift test --filter RestartPolicyTests
               -> 8 tests passed in 0.001s.
  LSP diagnostics -> clean across all 6 files.

Upstream alignment matrix (this PR vs apple#1258)
-----------------------------------------------------------

  RestartPolicy shape           bare enum         match
  unless-stopped mode           absent            match
  on-failure:N retries          absent            match
  restartPolicy optionality     non-optional      match
  decodeIfPresent default       .no               match
  Flag binding                  ExpressibleByArg  match
  RuntimeStatus -> ContainerStatus rename
                                NOT in scope      defer

The RuntimeStatus rename and the .restarting/.bootstrapped state
additions stay deferred -- they belong to the daemon-side enforcement
work, which is what apple#1258 actually does. This PR remains data-shape
only on the fork side; enforcement arrives when apple#1258 lands upstream
and the fork bumps its sync.

Refs CHAOS-1321, CHAOS-1385.
Sponsors apple#1258, apple#286.
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Code Coverage

Tier Line Coverage
Unit 24.76%
Integration 70.7%
Combined 79.46%

@jglogan jglogan added facet/runtime Work related to container lifecycle management and removed facet/cli Work related to generic or cross-cutting CLI features labels Nov 14, 2026
public var volumes: [String] = []

public func validate() throws {
if remove && restart != .no {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an example of the scattered validation I mentioned.

}

let state = try self._getContainerState(id: id)
try await self.lock.withLock(logMetadata: ["acquirer": "\(#function)", "id": "\(id)"]) { context in

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a performance note, I don't think it needs to block this PR.

kill and stop holding the lock will now block all other container operations, and these can take a while.

])
}

let options = try getContainerCreationOptions(id: id)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now runs before the state is marked stopped.

If it throws, the container stays running after its VM is gone.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's no case this throws.
Already existing ContainerCreateOptions doesn't have restartPolicy field, setting it default .no on deserialization.
Later creating ContainerCreateOptions with invalid fields throws validation error.

try? await self.resetBackOff(id: id)
}
)
try await stabilityMonitor?.track(id: id) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stability timer is registered before bootstrap and startProcess, while the PR description says the callback registers on a successful startProcess. If the VM takes longer than the timer to boot, resetBackOff sees a status other than running, skips the reset, and the backoff keeps doubling.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VM boot taking longer than 10 seconds (i.e., stabilityCall) is abnormal, and I'd handle this later if this is not a critical issue.

return ExitStatus(exitCode: 0)
}

try await bootstrap(id: id, stdio: [FileHandle?](repeating: nil, count: 3), dynamicEnv: [:])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dynamicEnv: [:] is problematic. It means restart of containers using --ssh will be broken, as will anything else that needs to plumb in a value can change between container create and run.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this, we'd need in-memory structure to remember ssh env per container.
Though, we need on-disk structure if we need to remember that across daemon lifecycle.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not support --ssh for restartable container.

Containers move to .bootstrapped state after assigning SandboxClient.
Following invariant holds:
 1. If .stopped, state.client = nil
 2. If .bootstrapped, state.client != nil.

It'd better to tightly couple the state and client.
Then, add `restarting` state to ContainerStatus
ContainersService spawn an unstructured Task for restart scheduler,
which receives stopped container ids through a async queue and restart
them if matching conditions.

While it doesn't run under lock, it doesn't introduce a new level of race, as
all the actual operations changing container state (e.g., `bootstrap`)
are lock protected.
This reuses ExitMonitor, which was not initially for this. It might be
better to come up with generic Monitor structure?
doStop cannot guarantee the container is stopped in short term.
The catch block's kill could throw out of the unstructured Task, which
triggers NoUseUnstructuredThrowingTask under warnings-as-errors. Log the
kill failure instead.
manualStopped was never cleared, so a stop followed by a start disabled
the restart policy for good. Clear it in bootstrap.

stop on a restarting container had no client to stop, so the scheduler
still bootstrapped it after the backoff. Move it back to stopped so the
scheduler's status check bails.
The CLITest framework was removed on main. Move the restart policy tests
to ContainerFixture under Tests/IntegrationTests/Run as a Serial suite.
…rt policy

Parse --restart as no, always, on-failure or on-failure:0 (unlimited
retries, matching Moby). Reject Swift enum spellings and non-zero retry
counts. Reject --rm combined with any restart policy other than no, both
in the CLI flags and in ContainersService.create. Add tests for parsing,
legacy options decoding, and the --rm conflict.
RestartPolicy.onFailure now carries a maximum retry count, where 0 means
unlimited as in Docker/Moby. The restart scheduler counts automatic
restarts and stops once the limit is reached; a start that does not come
from the scheduler resets the count. Add parsing, validation, and
integration tests for the retry limit.
Make ContainerCreateOptions.validate private and call it from the
throwing init and from init(from:), so an invalid combination such as
--rm with a restart policy cannot be constructed or decoded. Drop the
now redundant checks in the CLI flags and ContainersService.create.
Replace the failable ExpressibleByArgument conformance with a transform
closure so the specific RestartPolicy parse error reaches the user.
Update testStabilityCall for the longer backoff before run 8 (12.8s).
Only kill a container on restart failure once the scheduler has begun
bootstrapping it. A failure in restart() (for example because the user
already started the container) must leave it untouched.
kill previously marked the container as stopped by the user before
signalling, for any signal and any process. Set the flag only after the
kill succeeds, for the init process, and only for SIGKILL, SIGTERM,
SIGINT and SIGQUIT.
Keep at most one restart task per container, cancel it when the user stops
or kills the container, and bail out of bootstrap if the task was cancelled
while waiting for the lock.
The SSH agent socket is passed to bootstrap per start through dynamicEnv,
which the restart scheduler cannot supply. Fail bootstrap with an
unsupported error when a container with ssh enabled has a restart policy
other than no, rolling back to stopped so a scheduled restart does not
linger in restarting.

Also await restartTasks in kill, which accesses the actor-isolated
property from the lock closure, and document --restart and its
limitations in the command reference.
A bootstrapped container is transient, and before restart policies it was
just a stopped container with a runtime client. Do not expose it as a new
public status. Represent it as stopped with a live client instead:

- bootstrap leaves the status as stopped (also when coming from
  restarting), so stop tears the runtime down.
- handleContainerExit only returns early for stopped containers without a
  client, so a failed start is still cleaned up by stop.
- restart refuses containers that already have a client, so the scheduler
  cannot take over a container the user is starting.
kill failed with "no runtime client exists" on a restarting container
and left the restart scheduled. A terminating signal to the init process
now cancels the pending restart, as stop does: mark the container as
stopped by the user, clear the backoff, and move it to stopped without
signalling a process that does not exist.

For any other container the kill is unchanged, and the user stop is
still recorded only after the kill succeeds.
delete did not touch the restart task, so deleting a restarting container
left its task running until the backoff ended, and a restart could race
with the cleanup of a force-deleted one. Cancel the task in cleanUp, which
every removal path goes through, so it stops before the container is gone.
The restarting status is new, and existing API clients do not know it.
Degrade it to stopped in list, the only path that sends container
snapshots to clients, so the API stays backward compatible. The degrade
runs before the status filter, so a restarting container also matches a
stopped filter, consistent with what clients see.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

facet/runtime Work related to container lifecycle management

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Request]: Add support for --restart flag for run

5 participants