Codex harness troubleshooting

Symptoms, causes, and fixes for the Codex app-server harness. Part of the Codex harness guide; Where each section moved lists every section.

Troubleshooting

Codex does not appear as a normal /model provider: expected for new configs. Select an openai/gpt-* model, enable plugins.entries.codex.enabled, and check whether plugins.allow excludes codex.

OpenClaw uses the built-in harness instead of Codex: confirm the effective route is an exact official HTTPS Platform Responses or ChatGPT Responses route, has no authored provider request override, and that the Codex plugin is installed and enabled. Affirmative reasoning support and native reasoning-effort metadata do not count as request overrides. Headers, request parameters, timeouts, and payload compatibility switches still do: Codex declares an OpenClaw fallback that preserves the exact request, including for explicit runtime selections. Other unsupported routes/authentication and missing explicit harnesses stop execution with an error. The openai/gpt-* prefix and agentRuntime.id: "codex" alone are not execution proof; inspect the actual harness in the completed result. See Runtime selection.

OpenAI Codex runtime falls back to the API-key path: collect a redacted gateway excerpt that shows the model, runtime, selected provider, and failure. Ask affected collaborators to run this read-only command on their OpenClaw host:

(  pattern='openai/gpt-5\.[45]|openai[-]codex|agentRuntime(\.id)?|harnessRuntime|Runtime: OpenAI Codex|legacy OpenAI Codex prefix|candidateProvider[": ]+openai|status[": ]+401|Incorrect API key|No API key|api-key path|API-key path|OAuth'   if ls /tmp/openclaw/openclaw-*.log >/dev/null 2>&1; then    grep -E -i -n "$pattern" /tmp/openclaw/openclaw-*.log 2>/dev/null || true  else    journalctl --user -u openclaw-gateway --since today --no-pager 2>/dev/null \      | grep -E -i "$pattern" || true  fi) | sed -E \    -e 's/(Authorization: Bearer )[A-Za-z0-9._~+\/-]+/\1[REDACTED]/Ig' \    -e 's/(Bearer )[A-Za-z0-9._~+\/-]+/\1[REDACTED]/Ig' \    -e 's/(api[_ -]?key[=: ]+)[^ ,}"]+/\1[REDACTED]/Ig' \    -e 's/(OPENAI_API_KEY[=: ]+)[^ ,}"]+/\1[REDACTED]/Ig' \    -e 's/sk-[A-Za-z0-9_-]{12,}/sk-[REDACTED]/g' \    -e 's/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/[EMAIL-REDACTED]/g' \  | tail -200

Useful excerpts usually include openai/gpt-6-astra or openai/gpt-5.6-luna, Runtime: OpenAI Codex, agentRuntime.id or harnessRuntime, candidateProvider: "openai", and a 401, Incorrect API key, or No API key result. A corrected run should show the OpenAI OAuth path instead of a plain OpenAI API-key failure.

Legacy Codex model refs config remains: run openclaw doctor --fix. Doctor rewrites legacy model refs to openai/*, removes stale session and whole-agent runtime pins, and preserves existing auth-profile overrides.

The app-server is rejected: use Codex 0.149.0 or newer. Older, malformed, and unversioned servers are rejected. Newer semantic versions continue with a compatibility warning and normal runtime validation against the Codex version OpenClaw ships. Update or remove custom, remote, or desktop binary overrides that select another version.

/codex status cannot connect: check that the codex plugin is enabled, that plugins.allow includes it when an allowlist is configured, and that any custom appServer.command, url, authToken, or headers are valid.

An app-server message cannot be parsed: OpenClaw recovers raw newlines inside otherwise valid JSON strings. Invalid escapes or unescaped control characters produce a redacted warning, then decoding resumes with the next message on both Node and Bun. These invalid fragments do not consume the following valid message.

The resident catalog reports a spawn failure: a missing executable (ENOENT), missing execute permission (EACCES), or incompatible CPU (EBADARCH, sometimes shown as macOS errno -86) stops that catalog's automatic retries and records one advisory. This includes native launch errors whose diagnostics arrive after the managed launcher exits during registration or initialization, on both Node and Bun. Repair the installation, then restart the Gateway to retry. Unrelated configuration reloads do not retry the failed executable. Disabling the plugin through config reload retires its catalog refresh loop.

Managed passive catalogs use the plugin's installed Codex package and do not fall back to macOS desktop app bundles. OpenClaw runs its launcher with the Gateway's interpreter so Codex selects the platform package matching that interpreter's architecture without a node PATH lookup. At debug or trace log level, Codex app-server spawn records the executable, launcher, and resolved native binary paths without arguments or credentials. Use file <path> on macOS to inspect the failing executable's architecture. Check the managed native binary with openclaw doctor --lint --only codex/managed-app-server --json, including when Codex is enabled only for its session catalog.

Directories named openclaw-model-catalog-* contain OpenClaw plugin source captures, not native Codex sessions. Current captures are bounded by worker generations and retired with their workers. Doctor reports older captures outside managed custody with an offline cleanup command; it does not delete those legacy directories automatically.

The Codex app-server uses too much memory: distinguish the two processes first. OpenClaw runs the local Codex app-server as a separate Rust child. NODE_OPTIONS=--max-old-space-size=... changes only the Gateway's Node.js V8 heap; it does not cap or enlarge Codex. Managed Gateway installs already choose an adaptive V8 heap, and raising it can leave less host memory for Codex. Use Gateway memory troubleshooting for Gateway pressure, and inspect host or container memory for the Codex child.

Large catalog replies use a decoder worker in the Gateway. After a complete reply, OpenClaw releases that worker after one minute without further worker decoding. The native app-server connection and its warm conversation threads stay connected. A later large reply starts a new decoder, so its first response can take longer; small replies do not require a worker. Incomplete replies retain their decoder until recovery completes or the connection closes.

"Cannot inspect Codex processes": this error comes from local process inspection before model inference. For a deadline error, retry after host responsiveness recovers. For a permissions error, check access to /proc on Linux or ps on macOS.

A new turn fails during cleanup after a Gateway restart: OpenClaw checks registered Codex processes before starting a replacement. Registrations for processes that have exited or whose PID has been reused are removed automatically, without scanning unrelated processes. Boot cleanup and new connections serialize recovery so they cannot stop or resume the same orphan concurrently. Live orphaned processes still require verified descendant cleanup before replacement work starts. If cleanup remains blocked, check Gateway logs and host process-inspection access; do not delete process registrations to bypass recovery.

The bundled Codex has no heap or RSS limit and no configurable idle-unload delay. After the last client unsubscribes, an inactive thread can remain loaded for up to 30 minutes. OpenClaw independently keeps up to 64 idle conversation threads subscribed on each Codex app-server for 30 minutes after their last activity. This preserves warm sessions and session-scoped approvals when several conversations alternate. Active turns and parents with unfinished native subagents are protected from idle eviction; session reset or deletion releases its own thread immediately. Idle-limit eviction unsubscribes the least recently used conversation, after which Codex applies its separate unloading delay and a later resumed session can require approvals again.

On constrained hosts, reduce native Codex subagent fan-out before increasing the Gateway heap:

{  plugins: {    entries: {      codex: {        config: {          appServer: {            args: ["-c", "agents.max_threads=3", "app-server", "--listen", "stdio://"],          },        },      },    },  },}

That setting limits native child threads for the bundled Codex default multi-agent backend. If you explicitly enable Codex multi-agent v2, use features.multi_agent_v2.max_concurrent_threads_per_session=3 instead; the v2 limit includes the root thread and cannot be combined with agents.max_threads. For more Codex headroom, increase the host, container, or cgroup memory allocation. An OS hard limit can terminate Codex rather than backpressure it.

Model discovery is slow: check the app-server's connectivity to its model catalog endpoint. The default plugins.entries.codex.config.discovery.timeoutMs is 10 seconds so Codex can finish its native refresh or fallback. A shorter override can interrupt that fallback and make native models unavailable. See Codex harness reference.

Codex plugin state has reached its row limit: run openclaw doctor to check for bindings left behind by deleted or expired OpenClaw sessions. Stop the Gateway, then run openclaw doctor --fix to remove proven orphaned session bindings after session repair. Doctor preserves supervised bindings, active leases, ambiguous ownership, and bindings whose session store cannot be read. This cleanup does not delete native Codex thread history or managed-thread advisory records.

WebSocket transport fails immediately: check appServer.url, authToken, headers, and that the remote app-server speaks the same Codex app-server protocol version. Codex WebSocket transport remains experimental and unsupported; prefer managed stdio or the local Unix control socket.

Native shell or patch tools are blocked with Native hook relay unavailable: the Codex thread is still trying to use a native hook relay id that OpenClaw no longer has registered. This is a native Codex hook transport problem, not an ACP backend, provider, GitHub, or shell-command failure. Start a fresh session in the affected chat with /new or /reset, then retry a harmless command. If that works once but the next native tool call fails again, treat /new as a temporary workaround only: copy the prompt into a fresh session after restarting the Codex app-server or OpenClaw Gateway so old threads are dropped and native hook registrations are recreated.

Codex tool calls create too many short-lived hook processes: set plugins.entries.codex.config.appServer.loopDetectionPreToolUseRelay: false and restart the gateway. This disables only the Codex PreToolUse subprocess used for OpenClaw loop detection and its no-policy marker. Required before_tool_call and trusted-tool policy relays remain enabled.

A non-Codex model uses the built-in harness: expected unless provider or model runtime policy routes it to another harness. Plain non-OpenAI provider refs stay on their normal provider path in auto mode.

Computer Use is installed but tools do not run: check /codex computer-use status from a fresh session. If a tool reports Native hook relay unavailable, use the native hook relay recovery above. See Codex Computer Use.

Image-heavy conversations exceed the request-size limit

request_too_large with Request exceeds the maximum size accepted by the Codex inference relay (33554432 bytes) means the request exceeded the relay's 32 MiB body limit. This includes the decompressed body, so a smaller compressed upload can still fail. It is not a disk-space limit or a universal OpenAI model limit.

Inline images can consume many bytes while using comparatively few model tokens. Native Codex compaction uses token pressure, so image-heavy history can hit this transport limit first. Remote compaction also sends history to the model and can fail for the same reason. A completed compaction does not guarantee that the next request's retained images fit the relay limit.

Retrying unchanged history, restarting the Gateway, or deleting downloaded image files does not shrink inline image data already in the native conversation. The OpenClaw maxActiveTranscriptBytes setting limits the size of the stored native transcript, which is checked when the thread is bound at startup; it is not a budget for a single inference request body. See compaction ownership.

Before more image-heavy work, keep originals as files and use smaller previews or focused crops when the task permits. Save text findings and job checkpoints as you go. These practices reduce growth but do not enforce a native request-byte budget.

If the conversation is already blocked, preserve the saved work and prepare a text handoff from existing checkpoints. Try native /compact only if it is available for the selected route. If compaction is unavailable or fails with the same size error, an operator can start a fresh conversation with /new and supply the handoff without replaying the old inline images. Before resetting, check active/background work and let it finish or deliberately stop it; a reset can cancel work. Reconcile completed tool actions and deliveries before resuming. This is recovery, not a permanent prevention fix.

Worker-hosted app-server

The opt-in plugins.entries.codex.config.appServer.workerHostedCloud setting runs the pinned Codex app-server on an enrolled worker instead of running the model process on the Gateway. It requires a current Full remote-exec placement and the exact versioned codex.app-server.stdio.v1.<version> node command. Allow that exact command through the existing Gateway node command policy. There is no local fallback when the worker cannot start.

Provision a worker-only private bundle under the Gateway's configured state directory:

• worker-provisioning/codex/config.toml: the native configuration.
• One Node .mjs auth helper in that same directory, referenced by its absolute path exactly once in the selected provider's auth arguments.

Use directory mode 0700 and file mode 0600 on POSIX. Symlinks, nonregular files, malformed configuration, missing helpers and inputs larger than 16 KiB are rejected before pairing credentials are issued. Windows uses its existing private worker setup; POSIX mode-bit qualification does not attest Windows ACLs. The selected provider must use Responses with command = "node" and that helper as its sole argument, without competing provider credentials.

The provisioning owner derives the command's 64-hex identity from the configuration and helper bytes; relocating the helper into the lease preserves that identity. There is no separately supplied version or environment selector. These are trusted operator-authored executable deployment inputs, separate from the Gateway's CODEX_HOME, local auth profiles and session data. Missing bundle means opt-out; an incomplete bundle fails closed. Publish both files while the Gateway is stopped, then restart and reprovision affected workers when changing them. Existing lease replay rejects changed bytes or identity; runtime verifies the enrolled content. Enrollment delivers the bundle privately into the lease's codex-runtime directory only when the prepared node runtime includes the Codex plugin. Other worker runtimes neither load nor receive the private bundle, and image preparation does not capture it.

Only dedicated ephemeral, non-container nodes retain their platform-managed identity transport and CA settings. Ordinary paired nodes and container workers do not inherit these credentials. Gateway auth profiles and local GitHub identity environments are not forwarded. Selected Gateway auth profiles and native supervision bindings cannot be reused for worker-hosted inference. Clear the profile selection or detach supervision before choosing worker-owned auth. The selected Platform or subscription route still constrains the worker account; a mismatched account is rejected without logging the worker into a Gateway account. When a managed identity requires the host-owned tool surface, those tools continue to own command execution; this setting does not grant native commands access to that identity.

Native hook policies still require the existing remote hook callback deployment (appServer.nativeHookRelay). A worker connection is not a Gateway-local process. Private workspace resources gate file/process effects, not the initial tool declaration. Cancellation closes the invocation and joins its worker cleanup. Native thread state lives under the existing workspace HOME owner and can resume while that workspace remains available. This mode does not provide retained-disk recovery, worker-loss history recovery, or a replacement workspace guarantee.