On this page
On this page
Gateway
Restart recovery
A graceful Gateway stop or restart records which sessions are working or waiting, then aborts executing work promptly. This includes accepted turns still preparing their runtime when shutdown begins. At the next startup, an interrupted root execution gets one ordinary system input asking the agent to inspect current state and continue. Recorded yield waits stay parked when their wake source survives; a lost source produces a visible failure instead. The transcript remains the agent's context; the Gateway does not reconstruct or replay the old execution.
If the interrupted turn left a managed workspace pending, its recovery input waits for the workspace owner to finish preparation before starting the turn. Setup failures still produce the ordinary actionable error; pending setup alone does not discard the resume input. Setup scripts require the recovered caller's current administrator authority. The restart marker remains durable until the resumed turn is admitted, so another restart during preparation can recover it again.
During graceful shutdown, interrupted embedded runs have a short abort-settle budget (normally two seconds) to deliver their terminal partial reply before their writers close. Already accepted transcript writes still drain. A provider that ignores cancellation stops blocking the stream consumer after this grace, but unsettled underlying work remains retained until it actually closes.
The recovery input stays intact in the agent transcript but is hidden from the
Control UI chat feed, including queued, live, and reloaded history. The continued
answer remains visible as its own turn. This is not the separate update outcome
(restart-sentinel), which remains visible with any actionable failure guidance.
Control UI status summaries use source-owned copy, not recovery/workflow instructions. Their complete summary (including its label) is one line of at most 64 characters; oversized summaries retain their details behind a disclosure. This budget does not truncate agent prompts, ordinary messages, actionable update outcomes, or the existing expandable context notices.
A crash or SIGKILL cannot write shutdown markers. Startup also recognizes the
existing persisted nonterminal run identity when it has no live execution owner
and its run started in a recorded, unrecovered Gateway crash lifetime. The existing
boot ledger supplies each lifetime's exact start and the next boot's start; session
update times and arbitrary freshness timeouts do not authorize recovery. A successfully
completed shutdown adds no crash interval; its own work uses shutdown markers. A forced
stop or incomplete shutdown also recovers unfinished runs from that predecessor, even
when other stores have markers.
Older orphaned runs are recorded as stopped for restart and are
not resumed. Missing boot history also prevents unmarked recovery. Files, external
actions, and tool effects may already have changed, so the agent checks them
before repeating work.
On upgrade from the retired restart-recovery subsystem (through 2026.10.9),
startup and openclaw doctor --fix also repair tombstoned root sessions, with or
without a recorded continuation. The repair backs up the original database,
clears obsolete execution claims, and warns with the session key. Pending final
delivery is retired with an unknown outcome rather than replayed under obsolete
sender authority; its original content remains in the backup. Existing transcript
content is unchanged. Eligible roots receive one ordinary resume input; manual
archives, provider review, and external harness ownership retain their normal gates.
Every retained fence and pending final must match the tombstone's recorded run or
generation before retirement. A different generation, missing ownership link, or
live execution leaves custody intact and prevents automatic resume of that row.
Doctor names the session and asks the operator to inspect its retained identities
and delivery outcome against the transcript before repairing custody. Legacy scalar
delivery fields can still be normalized without discarding the pending final, so
ordinary admission no longer depends on the retired tombstone gate.
For unmatched custody with no live execution owner, openclaw doctor --fix backs
up and retires the legacy recovery fields when the session is terminal or archived,
preserving its status, archive, current writer identities, and delivery notice
without resuming it or replaying a pending final; startup reports an aggregate
count per agent instead of repeating warnings for each such session.
After a clean shutdown, the Gateway waits for final cleanup and flushes file logs
and output pipes. On macOS and Linux, any remaining referenced handles get a
one-second final drain before the process logs remaining handle types and re-raises
SIGTERM. Failed or retained
cleanup keeps its existing deadline and exit-status handling.
Unexpected exits
The Gateway records each run in its existing SQLite boot lifecycle. A graceful
stop, drained SIGTERM, deployment handoff, or planned update restart closes the
run. If the next startup finds that its predecessor did not close, it records
one unexpected exit and logs previous Gateway run ended unexpectedly.
The last ten crash records remain visible in openclaw gateway status, the
health and status RPC results, and local or authenticated /readyz details.
The Control UI's Debug health and status snapshots show these same records.
Doctor shows a Fix now warning about any crash detected in the last 24 hours,
or at least three detected in the last seven days, with a next step to inspect the
recorded exits. This diagnostic does not trigger a repair or restart: a healthy
successor remains ready. An unauthenticated remote readiness probe does not disclose it.
Records include the previous PID, start time, and available version, build,
runtime, and heartbeat facts. After startup, the Gateway performs bounded
supervisor inspection for exit evidence. On systemd an indexed query reads structured
exit records for the previous service run. A second query, limited to the final
30 seconds and the previous PID and unit, checks for the exact self-recovery
marker when a stalled main thread killed itself. Both queries share a one-second budget. On launchd it records the
daemon's explicit last exit status and reason; an integer status is not treated
as evidence of a signal. Missing evidence stays cause unknown;
an ordinary fresh start is not evidence of a clean previous exit. The shared
SQLite machine-state KV holds bounded diagnostic metadata alongside the existing
boot receipt; there is no crash sidecar file or agent-database scan.
Installed supervisors handle process restarts using their existing retry and
crash-loop limits. Foreground openclaw gateway does not acquire an automatic
restart loop. See Gateway service and process troubleshooting
when crashes repeat.
A wedged main thread
If the Gateway main thread makes no progress for five continuous minutes, its
worker-thread stall observer writes main-thread stall recovery: to the service's
stderr log, including the PID, elapsed time, threshold, and signal=SIGKILL, then
kills the process. Five minutes leaves generous headroom over ordinary stalls
lasting tens of seconds. Every successful heartbeat resets the deadline; separate
stalls do not add up. If the observer itself was suspended (for example, during
laptop sleep), it starts a fresh observation window on wake. This bound is fixed, with no configuration option.
Systemd, launchd, and the macOS app's child supervisor restart the failed Gateway.
Eligible interrupted runs then follow the crash-recovery rules above. A foreground
Gateway or gateway:watch needs an external supervisor to restart automatically.
The observer uses SIGKILL because a wedged JavaScript thread cannot handle
SIGTERM; it avoids SIGABRT and its potentially large core dump.
Look for preceding main-thread stall in progress: lines for bounded JavaScript
profile evidence when the runtime supports it. Stack capture is best effort and
cannot hold up termination; Bun and uninterruptible native stalls may only have
the heartbeat diagnostic. These off-thread lines go directly to stderr: use the
systemd journal, launchd stderr log, or macOS app Gateway log rather than relying
only on openclaw logs.
Recovery is suppressed while an inspector is enabled, including debugger pauses, and when startup profiling or coverage flags conflict with diagnostics. The heartbeat observer works independently of the optional V8 flight recorder.
What survives a restart
| State | Behavior |
|---|---|
| Conversations and transcripts | Remain in the per-agent SQLite database |
| Running root session at graceful shutdown | Receives one resume input at the next startup |
| Root session left unfinished in a pending crash-recovery interval | Receives the same single resume input at the next startup |
| Accepted user input captured in that shutdown marker | Runs after the resume input, in FIFO order, with its original source limits and current authority |
| Running sub-agent | Stops quietly; its root requester receives the resume input |
Session parked in sessions_yield with a surviving wake source |
Keeps its recorded wait, including dashboard roots; no startup model request |
| Yielded requester whose descendants were stopped | Receives the root resume input to reconcile their work |
| Yielded session whose external continuation source is gone | Closes with a recorded, visible failure |
| Background process | Ends with the old process; its root session receives the resume input |
| Already-produced outbound messages and media | Keep the existing delivery queue and delivery reconciliation |
| Scheduled jobs | Keep cron's own scheduling, catch-up, and interruption policy |
| ACP or external-harness session | Keeps its harness owner's restart policy |
| Update outcome | Remains a passive report in the originating conversation |
| Gateway terminal PTY | Ends with the old process |
A session counts as running when its own agent loop is active, a descendant sub-agent is running, or it owns a running background process. The Gateway reads these facts from their live in-memory owners during shutdown. A child contributes to its top-level requester; the child does not receive an independent resume input. Idle and finished sessions are not marked. Cron occurrence sessions and ACP/external-harness sessions are excluded.
The marker is a small private field on the existing session entry. It records the IDs and original authenticated source limits of accepted user inputs still queued behind that session at shutdown. These private limits are not included in chat history or model messages. The Gateway writes one marker per running root session or recorded yield wait, batched by session store, before aborting work. It uses the existing session-entry JSON in SQLite and needs no schema migration.
Shutdown closes request admission and scheduled wakeups before writing markers. Accepted ingress inputs retain their live source authority until the markers are written and the session owner stops them for restart. A real grant revocation or permission change during that interval still cancels the affected work. At startup, queued inputs must regain current authority before they can execute.
Crash recovery uses the nonterminal run marker committed with accepted plain
chat input, before dispatch hooks or runtime start. The same commit records its
writer identity; terminal settlement clears the run marker. Recovery is bounded
by the recorded crash intervals. Command persistence
records activity only; commands gain recovery markers when their runs start.
Commands that become runs, such as /goal or skill rewrites, remain unprotected
between input persistence and runtime start. A plain message that a hook ends
without starting a run can retain its marker and be retried after a crash.
The same startup metadata read discovers both sources; recovery does not reconstruct execution.
Startup recovers ready stores independently. Each physical agent's database
admission owns its recovery pass, including delayed preparation and automatic
preparation retries. An unavailable database records resume deferred: agent database unavailable with its agent ID; it does not suppress another agent's
recovery. Recovery runs after database preparation ends, under the session's
current authority, before the same session accepts fresh inputs. Another agent becoming
available does not revoke an already admitted session's authority.
The boot owner retains each agent's pending crash intervals in the existing shared SQLite machine-state store until its recovery pass completes, including an empty successful scan. An intermediate process that never admits that agent cannot erase its recovery window, even if that process shuts down cleanly. Separate crash intervals remain separate: clean intervening lifetimes never become eligible by joining the oldest start to the newest end. Consecutive crash intervals coalesce. The pending record survives boot-history pruning and is removed after recovery or when the agent's database registrations are removed. There is no schema migration or new configuration. Older binaries ignore this record and cannot provide delayed cross-restart recovery; upgrading again does not replay run identities already consumed by recovery.
Refused databases keep their normal admission safeguards. Shutdown cancels pending recovery without consuming its markers or discarding its crash intervals. Recovery consumes each eligible persisted run identity once; retrying a retained interval cannot resume that same run again.
Startup opens external input after recovery discovery installs each session's admission wait. Recovery admission continues in the background: a fresh turn for a recovering session waits until its resume input and retained FIFO inputs own their ordinary queue slots. Other sessions in the same agent can proceed. Stores prepared later install these waits before publishing database admission. This also applies when several logical agents share the same physical store. Database reads and writes retain their normal admission rules; recovery does not borrow startup's temporary database access or wait for model executions to finish. Crash recovery waits for preparation to settle without an interactive request deadline. Slow preparation does not disqualify interrupted work; failed preparation still prevents unmarked adoption and reports the normal admission diagnostics. This requires the updated Gateway, with no marker or schema migration.
Gateway readiness does not wait for recovered turns to enter their queues.
Catalog and read-only methods such as models.list, mentions.list, and
sessions.github.options use their own data owners' readiness. Startup retries
in the Control UI back off from 500 milliseconds to eight seconds.
Graceful shutdown
Once a stop or restart is committed, the Gateway fences new admission, snapshots running sessions, writes their markers across independent stores in parallel, and aborts active runs. Marker persistence and subagent settlement each receive at most ten seconds within the existing shutdown deadline. Root cancellation does not wait for subagent settlement. If marker persistence times out, each eligible root's restart cancellation records resume intent in its existing terminal write. Runs that never finish that write remain eligible through the predecessor's persisted nonterminal identity. It joins accepted persistence before retiring state and gives other resource cleanup a bounded grace. A long model stream, tool call, or background process does not get a full run-completion drain before shutdown.
Optional session preview backfill stops scheduling when restart drain begins. Queued rows are dropped; an already-admitted database operation settles through the existing database drain before retirement. The next Gateway rebuilds any missing previews.
After marker persistence and cancellation, process-owned shutdown logs a warning when required connection cleanup exceeds five seconds, then keeps joining its accepted finalizers before releasing the Gateway lock and state admission. The existing process shutdown deadline still bounds this join. Session admissions and cron finalizers that settle after their grace warning still proceed through database retirement; the warning does not permanently abandon the close operation. Resume markers do not replace pending transcript or cron outcome writes. Embedded hosts also join those resources before reusing the runtime.
Plugin-origin finalizers retain shared-state admission through this drain so
worktree run leases and subagent settle records can persist before SDK retirement.
Final cleanup failures are reported as shutdown failures, not startup failures.
The failure summary names the fatal cleanup error, including nested resource-owner
causes; earlier grace warnings are listed separately as warnings. An admissions
grace warning includes a bounded pending snapshot: effect count and the first
pending effect's phase, source phase, operation phase, interruption and settling
status. The same facts appear in the pending close span. Session and run identifiers
and session content are omitted; none means the observed owner has no such phase,
and null means its boolean status is unavailable. A zero effect count does not
mean the full admissions join has settled: it also waits for source, subagent, and
terminal-persistence cleanup.
A plain stop (SIGTERM or SIGINT) arriving during an in-process restart makes
the Gateway finish its bounded shutdown and exit instead of starting again.
Managed update handoffs keep their own stop handling.
Failed startup uses the same cleanup grace when the CLI will exit. Usage backfill cancels pending discovery reads and scans when its sidecar stops; a later startup rebuilds any unfinished cache work. Embedded hosts and in-process startup recovery still join cleanup before reusing the runtime.
An external stop or restart logs bounded-grace leftovers, such as a service-child
cleanup deadline, lifetime sidecar, or retained plugin runtime, and returns with
status 0 unless required persistence failed. A runtime that still owns resources
is never reused for an in-process restart. Node exits naturally after owned handles
close; the native shutdown watchdog kills the process if leaked handles keep it
alive two seconds past the shutdown deadline. Its worker and main-thread fallback
share one monotonic deadline, including worker startup time, and report retained
cleanup owners before forcing exit. After a clean shutdown retires those
watchdogs, the command waits ten seconds past its own final cleanup. If a leaked
handle still keeps the process alive, the Gateway prints the active resource types
to stderr. On Linux and macOS, a stop with exit status 0 then ends with SIGTERM,
which supervisors count as a clean stop. A non-zero status, which can be a
supervisor restart instruction, and Windows keep the report only. A duplicate SIGTERM during shutdown is
logged once and ignored. That signal exit is accepted, and the supervisor starts
a fresh Gateway.
A previously recorded supervisor cleanup failure is reported as retained resources with its original owner details; it does not fail a later external stop. The Gateway still joins the supervisor before retiring database admission, read-only workers, and the spawn broker, even when that join rejects. A settled process-supervisor failure no longer prevents agent-database close; supervisor cleanup still pending past its grace retains those databases.
Reversible suspension and host freeze preparation keep their own semantics. A
suspension does not mark sessions or abort them. Preparation with drain: true
can become ready once persistence is clear; the committed stop or restart performs
the snapshot and abort.
Requests the Gateway did not accept before admission closed receive the ordinary retryable restart error. Clients retain their submission IDs and use normal reconnect handling. A disconnected RPC may already have performed its work; clients reconcile the existing receipt rather than replaying arbitrary RPCs.
Maintenance custody observations
Gateway status reports its process-owned shutdownBudget, with activeWork
counts and a separate writeCustody array. Suspension preparation and status
responses can include writeCustody entries with phase and count, covering
migration, backup, coordinator writes, session lifecycle mutation, and terminal
persistence. Ordinary root requests and cron runs do not by themselves imply write
custody. Counts can overlap.
Older Gateways can omit this optional distinction. Their running code keeps its old shutdown behavior until restarted; installing new files cannot shorten an already-running Gateway's drain.
Systemd stop deadlines
The Gateway inspects the running unit's effective TimeoutStopUSec, including
drop-ins, and caps cleanup to the supervisor's remaining stop budget. Inspection
and startup waits consume that same deadline. OpenClaw-managed units currently
request TimeoutStopSec=330; this is an outer safety allowance, not a request to
wait for agent work to finish.
Use KillMode=mixed so systemd's initial stop signal reaches the Gateway rather
than every child runtime. Systemd can still terminate remaining children after
the Gateway exits or its deadline expires. openclaw doctor --fix and updates
refresh outdated generated unit policy; operator-owned drop-ins remain the
operator's responsibility. Inspect them with:
Use the system manager without --user for a system service. If inspection fails,
the Gateway logs a warning and uses its conservative fallback budget. A failed
shutdown reread does not grant a longer timeout. See Linux services.
Launchd stop deadlines
The macOS Gateway reads the loaded job's effective exit timeout with
launchctl print, binding it to the Gateway or its launcher PID. The loaded job,
not the plist on disk, determines the deadline. Editing a plist or using
launchctl kickstart -k does not reload it; service replacement does.
Generated LaunchAgents request ExitTimeOut=330. Doctor can migrate the retired
20-second generated template after backing it up, but preserves customized values.
A launcher can impose a shorter child-reap deadline. Inspect the loaded job rather
than assuming the plist value is effective. See macOS Gateway.
Host sleep and process freezes
When a gateway host wakes from sleep, a virtual machine resumes, or the process continues after a long pause, the gateway detects the freeze within about 30 seconds. It restarts channel connections once tracked Gateway work is idle, then refreshes cached health and presence. The health and presence refresh still runs when a busy gateway defers only the channel restart. This keeps stale sockets from waiting for their normal expiry without interrupting an active reply or agent startup when a busy event loop caused the timer gap.
Sleep and backward wall-clock adjustments do not by themselves revoke state ownership. After a heartbeat gap, the Gateway verifies its lock's physical identity and acquisition bytes before renewing it. A removed or replaced lock, or a renewal that still fails, fences state access. Locally authorized recovery still restarts the Gateway when the fenced database cannot supply the stored restart intent; raw unauthorized restart signals retain their existing policy. No configuration or database migration is needed when updating to this behavior.
The macOS app and Linux companion cooperate with a local gateway by preparing a
short suspension lease before the host sleeps and resuming it after wake. Remote
gateways are not suspended when the app host sleeps. A deliberate suspension
through gateway.suspend.* is reversible: it neither writes restart markers nor
aborts running sessions. Work continues under the suspension owner after resume.
Recovery after a failed update
After a failed interactive update or repair, OpenClaw finishes cleanup and any
service recovery, then opens openclaw triage. Triage immediately
starts the first directly launchable coding agent in this order: Claude Code,
Codex, OpenCode, then Pi. It passes the captured failure before fresh Doctor
checks or archive collection and asks the agent to diagnose, repair, and verify
the installation. The agent receives the captured installation paths and keeps
its normal authentication, sandbox, and approval settings.
For a failed Control UI or unattended update, use the installation-specific
command printed on the Gateway host, or run triage there with the same OpenClaw
profile and state/config paths. Use --agent to select a particular coding agent:
JSON, --yes, and non-interactive update invocations collect diagnostics without
starting an external coding agent. openclaw triage --non-interactive also prepares
diagnostics without launching an agent. --update-result <path> includes an
updater's saved failure artifact. Printed handoff commands preserve installation
selectors and use PowerShell on Windows or POSIX shells on macOS, Linux, and WSL.
Staging and validation run while the old Gateway serves. The candidate runs
Doctor lint, config and plugin planning, and an isolated canary boot against
copied configuration and verified database snapshots. Migrations on these
copies rehearse the upgrade without changing live state. A validation failure
can enter bounded unattended repair
while the old Gateway keeps serving. Activation requires the failed check to
pass. Otherwise the candidate is discarded. An already-current no-op never
stops the Gateway.
Older targets that predate migration continuation record runtime validation as
unavailable and use the existing downgrade finalization path.
The detached helper also waits for the activating phase before parking its
parent Gateway. The first activation window contains the swap, required live
migrations, and service start. Plugin package download and sync run while the
core Gateway serves. A changed plugin snapshot requires a second measured
activation window: full Doctor migrations under exclusive maintenance, then
restart and verification. Unchanged plugins do not run another full Doctor pass.
After activation, the updater verifies that the managed service is running and
owns its port, the Gateway hello handshake matches the expected version/build
identity, health checks pass, plugins and channels are healthy, and
/readyz returns HTTP 200. Update verification does not use model inference.
Startup receives the update's existing per-step --timeout budget (1800 seconds
by default), including migration, listener initialization, and readiness checks.
On the first update from an older release, the old updater
invokes the newly installed CLI but does not pass that readiness budget. The
candidate recognizes the existing update marker and, once the managed process is
running, uses the five-minute startup watchdog instead of the standalone
60-second deadline. Migration, listener, and health transitions do not reset this
bound. The old updater's subprocess timeout also remains in force. An exhausted
wait reports the last observed startup phase. Standalone restarts use the progress-gated readiness wait.
Verification facts and measured downtime are retained in the
update run report.
When a package fails verification, the updater compares the shared and affected
per-agent SQLite user_version values and configuration content with their
pre-activation values. If they are unchanged and the previous runtime was
verified before activation, it restores the previous package, command shim,
service definition, and config writer stamp, then starts that runtime and repeats
the CLI verification checks. Successful recovery leaves that Gateway running
and finishes rolled-back, with the failing check kept as the reason and
downtime covering service stop through verified recovery. The writer-stamp guard
does not block this intentional recovery. Its allowance is scoped to rollback
service commands and never persisted. See
Automatic rollback.
If configuration content or a schema version changed, automatic rollback is
refused (state-migrated-no-rollback) and the updater attempts bounded repair on
the installed candidate. The same repair slot can run when rollback itself fails.
Code rollback cannot reverse state migrations. An unavailable schema comparison also prevents
automatic rollback (rollback-state-unverified). After migration,
a fresh candidate process finishes verification and the same durable run report.
The old updater does not reopen the newer database. Git activation failures before live migrations can restore
the previous source and retained built runtime. Later Git failures retain the
candidate for diagnosis.
An update failure does not by itself authorize a candidate restart. Candidate activation still requires successful validation. A blocking live Doctor result does not become a restart grant. The previous runtime was verified before the update, so rollback across unchanged configuration and schemas may restart it under that prior verification and must verify it again afterward. A detached helper or Windows task autostart cannot bypass this decision.
During post-activation repair, the orchestrator starts or restarts a stopped or
unhealthy service once after each turn, then reruns verification. Passing checks
allow the run to succeed. If rollback already restored the previous release,
successful repair finishes rolled-back and the command still exits nonzero.
Otherwise it fails with the original reason and repair summaries. The agent
cannot issue service lifecycle commands.
On Windows, captured Scheduled Task autostart stays suspended through Doctor finalization. The updater enables the task for activation and restores suspension if final verification fails, including after a migrated-state handoff. Native task-control failures appear in the update report. Failed suspension never triggers automatic re-enablement of the rejected installation.
On macOS, a terminated update helper can leave the selected Gateway LaunchAgent
installed but unloaded and disabled across logins. openclaw doctor and
openclaw doctor --fix diagnose this state. After successful standalone repair,
--fix starts and verifies an already-stopped managed Gateway whose service
targets the current installation. Update-time Doctor leaves activation with the
updater. If the update was interrupted or installation safety is uncertain,
rerun openclaw update or use Doctor and triage before starting it manually.
Once verified, run openclaw gateway start (or
openclaw --profile <profile> gateway start) to re-enable and start that service.
Keep the same state/config and custom-label overrides. Doctor prints the selected
label and recovery command. Interactive Doctor can offer bootstrap repair.
A cancellation before package mutation can restore the original service under its existing handoff ownership. Recovery succeeds only after the Gateway passes the normal restart health checks and reports the verified installation version and, for Git recovery, the exact restored build ID. A matching package version alone cannot distinguish two Git builds. A service manager accepting a start request, or reporting a live PID, is insufficient. Once the detached helper launches the updater, a missing, malformed, oversized, or interrupted direct result leaves activation to the operator. This is stricter than older helpers that restarted after an unclassified failure. Installing a new target does not change an already-running historical helper. These checks apply to the helper version that started the update.
A skipped update before activation does not park or restart the Gateway. If an interruption occurs after parking, the helper uses the child's verified recovery decision and preserves the original reason. A zero exit is retained only if required recovery succeeds or the child already verified it.
Updater exit code 79 keeps the Gateway parked only when the previous generation
cannot be safely restored and verified. When the updater has restored the previous
generation across unchanged configuration and schemas and supplies a verified
recovery decision, the helper starts and verifies it instead of leaving it
stopped. Helper recovery verifies service liveness, version/build identity,
plugin activation, and channel health. It does not repeat the separate /readyz
check. That report field remains unverified.
The run then finishes rolled-back with the previous version and measured
downtime. Missing recovery proof, migrated state, or failed restoration still
requires repair before restart. A service that is observed stopped is recorded
as stopped. The report does not reuse its pre-activation running status.
A terminal failed update still exits nonzero when later service recovery or triage repair succeeds. Error and skip notifications are attempted before recovery. The helper does not recreate them after the recovering Gateway consumes them. Check the final CLI result and the handoff log for the recovery outcome.
Repair the failed Doctor or installation check before restarting. Triage can
inspect openclaw gateway status --deep and the update diagnostics. Avoid blindly installing
older code after a newer release has migrated configuration or databases. See
Updating and recovery. Restart sentinels report the outcome.
Copying one does not grant permission to restart a service.
Automatic resume
On the first upgrade from a version with legacy main-session recovery, startup
adopts an unsettled cycle only when the entry has abortedLastRun: true, status
interrupted, and an updatedAt within the six hours before this process started.
Old shutdown marking updated that timestamp. Startup clears the cycle and sends
the same single resume input under the current-authority rules below. Stale,
future-dated, settled, done, and killed entries do not resume through that bridge.
Legacy tombstones use the backed-up repair described above and do not require
a recent timestamp or a continuation session.
An installation stopped for longer than six hours before upgrading loses this
one-time resume; send a new message to continue. Old execution and authority
snapshots are not restored.
Startup consumes the shutdown marker and orphaned run identity together and admits one new input through the normal session-event path:
The Gateway restarted while you were working. Resume your work: check the current state of anything that was in progress (files, processes, subagents, external actions) before repeating it, then continue.
If the session had running background exec processes, that same input also says:
Background exec processes were stopped; rerun them if still needed.
This is a new input, not a replay of the original user prompt or unfinished tool call. Later model requests can contain the text as conversation history. The agent checks the transcript and current state before continuing; effects are not guaranteed exactly once.
For a session with a human owner, the input uses that recorded principal's current
role, scopes, and session access, narrowed by the session's current permission mode
and tool policy. When no owner is set, the human creator is used. On a Gateway
without gateway.roles, a profile has no authority of its own: the input uses the
authenticated connection the person last sent from in that session (its device,
sign-in method and scopes), revalidated against current pairing, auth and profile
state, and never more than that connection had. A person who never sent from an
authenticated connection in that session cannot be resumed there. A removed
principal or lost access prevents automatic resume. An admission failure records
a visible interruption notice in the existing session, with a bounded, redacted
diagnostic. The warning identifies the session, interrupted run, and admission
stage. Resolve the reported error, then send a new message to continue. Archived
or deleted sessions remain skipped with a log entry.
The Gateway never substitutes an administrator. A recorded connection is only a basis for
revalidation, never a saved grant.
Sessions with no human principal use ordinary system-event authority; if it lacks
a needed capability, the agent must ask for an authorized input.
WebChat uses the same session and transcript. An unanswered accepted user prompt stays in history as its original entry, separate from the restart input. Channel sessions use their current delivery route so normal replies return to the conversation. The resume input inherits no old browser connection, approval, or requester authority snapshot; on a Gateway without roles it uses only the current, revalidated authority of the person's recorded device.
Only accepted user messages whose IDs were captured in the graceful-shutdown marker follow the resume input, in FIFO order. Older queued rows left by a crash remain inert, even if the session is marked during a later shutdown. Each retained message retains its original app grant, ID membership, and scope and model limits. The Gateway revalidates that source against current policy before replay; its live authority remains required at tool effects. A later sign-in by the same person cannot replace the accepted source. Missing, unsupported, or revoked sources produce an ordinary failed send that the sender can resend. This also applies to older shutdown markers without per-input source evidence. A channel identity without a profile does not authorize queued execution after restart.
New messages arriving during a resumed run use ordinary steering and queue rules.
Resumed turns execute on the bounded restart-recovery lane at
agents.defaults.maxConcurrent, independently of interactive and cron capacity.
A recovery backlog cannot queue ahead of new cron runs. Per-session input order
is unchanged; Stop, cancellation, archive, and reset keep their normal meanings.
Subagents
Graceful shutdown marks the root requester of a running child, then cancels the child through the ordinary abort path. The child settles terminally without a shutdown completion announcement or a "subagent failed" notice to its parent. Children are not adopted or independently resumed at startup.
A child that already ended its execution with sessions_yield waitFor:"message"
keeps its paused task across restart. Shutdown marks that session directly;
startup consumes the marker while preserving the registry's exact paused task,
without replaying execution or restoring old tool or completion authority.
The same registry fact protects a recorded message wait after an unclean stop.
A fresh continuation from its controlling requester goes through ordinary
admission and retains task-owned result delivery. If the requester finishes or
is released without continuing it, the wait closes with a recorded outcome.
Ordinary ownerless mid-turn runs still follow the orphan recovery rules above.
A root or nested requester already parked in sessions_yield retains its exact
yielded lifecycle when its recorded child dependency can still complete. This
includes dashboard/operator sessions; existing sessions_yield rules still
determine which completion they await. Startup
does not send a resume input or run the model for that wait. A fresh continuation
of the surviving child delivers its completion through the ordinary requester
handoff. Each parked session has one bounded startup log line with its agent,
session key, and yield-waiting reason. The restart summary's parked count
describes these retained waits, independently of conversation environment cleanup.
A requester yielded for children that restart stopped keeps the ordinary root recovery path. Its single resume input reconciles the stopped children's work; it is not re-parked waiting for their old executions. A stopped child also needs reconciliation when other children still have reachable waits.
When an external continuation source is gone, such as a requester that finished or was released without continuing its message-waiting child, the wait closes with a visible recorded failure. Its transcript remains available. A permanently failed completion handoff likewise closes its original parked requester with classified guidance to inspect saved child work and retry or resend. It cannot close a newer turn or undo Stop.
A parent's single resume input covers reconciliation of its children. It can inspect their transcripts and results with the existing sub-agent tools, then send a new instruction or spawn replacements after checking what already happened. See Sub-agent operations.
Agent-requested restarts
The restart sentinel keeps the update or restart outcome and its originating route. It can append the passive outcome report or deliver already-produced bytes; it does not launch another model turn or replay an update continuation. If the requesting session was running at graceful shutdown, the generic marker sends its one resume input.
Update sentinels retain their stats.runId link to update_runs. The new Gateway
records observed version/build and startup facts without replacing a terminal
outcome already written by the updater. The originating session keeps the report;
consuming a sentinel does not remove update history. See
Update reports.
The sentinel's SQLite columns remain authoritative. Retired restart-sentinel.json
and .doctor-importing files are not runtime inputs. Installations requiring that
old import must upgrade through 2026.9.7 and run openclaw doctor --fix first.
Legacy delivery-queue rows requesting inference remain inert; they do not grant
permission to run a model.
Safety valves and observability
The crash-loop breaker still suppresses automatic side-service startup after three unclean boots within five minutes. The control plane remains available for inspection. A stable process rechecks after the full window drains and starts eligible deferred channels; manually stopped channels remain stopped.
If the Gateway is reachable but channels stay down, inspect
openclaw gateway status --deep and openclaw channels status --probe, fix the
startup failure, and use openclaw doctor as needed. An authorized operator can
manually override suppression for one channel:
This override does not disable the breaker for other channels. Session resume uses the ordinary input path and does not reconstruct the interrupted execution.
Startup migration warnings remain visible in status and Doctor. Unsafe required
state still prevents startup. Run openclaw doctor --fix against the same state
and configuration before restarting when the diagnostic asks for repair.
Marker persistence, skipped authority, and bounded shutdown leftovers are reported in logs. A resume admission proves only that the new input was accepted; use the session transcript and actual delivery outcome to check whether work finished.
Apart from that upgrade bridge, old restart claims, authority snapshots, interrupted pending-input rows, and queued inference records may remain in existing databases. They are ignored rather than reconstructed or deleted. Their text stays preserved. Legacy interrupted and ownerless queued inputs left by a crash do not add pending counts, notices, or history pins. Normal retention applies to data that was pinned only by those retired recovery records.
Verify recovery after an update
Confirm the Gateway is ready, then inspect each previously running root session. It should have one resume input and the resulting work or an actionable ordinary error. Idle sessions should have no new model turn. Check queued user messages for normal transcript consumption or a failed-send outcome. A healthy process alone does not prove that a reply reached its channel.
What is not resumed
- Work that never persisted a run identity before a crash or
SIGKILL. - Idle or finished sessions.
- Sub-agent children independently of their root requester.
- Cron occurrences and ACP/external-harness work, which keep their own owners.
- Deleted or archived sessions, and human-owned sessions whose current principal cannot be admitted.
- Gateway terminal PTYs and background OS processes.
- Legacy inference queue entries or settled old execution-recovery records.
The stored conversation remains available after an unclean stop. If no resume input was admitted, send a new message to continue and inspect uncertain effects before repeating them.
Automatic peer results
Ordinary live sessions_send communication and its current authorization remain
unchanged. Restart does not restore a peer sender, replay a requester-settle input,
or launch a new model turn from a saved completion. Already-produced outbound
messages and media retain their delivery owner. A running root session's resume
input is responsible for inspecting pending child or peer work.