Skip to content

Commit a2b7847

Browse files
Viperdk2020Viperdk2020
committed
docs: document memory backends
1 parent 0e37094 commit a2b7847

3 files changed

Lines changed: 98 additions & 55 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,6 @@ You can also use Codex with an API key, but this requires [additional setup](./d
6262

6363
Codex CLI supports [MCP servers](./docs/advanced.md#model-context-protocol-mcp). Enable by adding an `mcp_servers` section to your `~/.codex/config.toml`.
6464

65-
6665
### Configuration
6766

6867
Codex CLI supports a rich set of configuration options, with preferences stored in `~/.codex/config.toml`. For full configuration options, see [Configuration](./docs/config.md).
@@ -71,16 +70,20 @@ Codex CLI supports a rich set of configuration options, with preferences stored
7170

7271
Codex keeps a lightweight, per‑repository memory of key actions to help you recall decisions and changes in each project. Entries are written locally to `<repo>/.codex/memory/memory.jsonl` after tool use (shell exec, MCP tool calls) and patch application. The file is plain JSONL so you can search, back up, or clear it easily. See [Per‑repo memory](./docs/config.md#per-repo-memory-local) for details.
7372

73+
For backend choices (JSONL vs SQLite) and maintenance commands, see [memory backends](./docs/memory-backends.md).
74+
7475
Toggle per run (TUI or exec): add `--memory off` to disable, or `--memory on` to force‑enable. You can also set `CODEX_PER_REPO_MEMORY=0|1`.
7576

7677
Durable memory and smarter preamble
78+
7779
- Durable items: In addition to action logs (exec, tool, change), Codex records durable items you and Codex can reuse across turns:
7880
- `pref`: your preferences and decisions (e.g., “Prefer Ruff + Black”).
7981
- `summary`: short outcomes/facts captured on task completion.
8082
- `decision`: approval checkpoints and key decisions.
8183
- Smarter preamble: On new sessions, Codex injects a short “project memory” preamble built from recent `pref` and `summary` items. It deduplicates, merges tags, caps counts, and enforces a length limit so your prompt stays concise.
8284

8385
TUI: memory slash commands
86+
8487
- Add preference: `/memory add Use Ruff + Black`
8588
- List recent durable: `/memory list 10`
8689
- Search durable: `/memory search ruff`
@@ -89,6 +92,7 @@ TUI: memory slash commands
8992
- Help: `/memory help`
9093

9194
What gets logged automatically (TUI)
95+
9296
- On task complete: a `summary` durable item with a compact preview of the last assistant message (kept brief).
9397
- On approval request: a `decision` durable item noting the request (exec/patch).
9498

‎docs/config.md‎

Lines changed: 57 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
# Config
22

3-
43
Codex supports several mechanisms for setting config values:
54

65
- Config-specific command-line flags, such as `--model o3` (highest precedence).
@@ -388,10 +387,10 @@ set = { CI = "1" }
388387
include_only = ["PATH", "HOME"]
389388
```
390389

391-
| Field | Type | Default | Description |
392-
| ------------------------- | -------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
393-
| `inherit` | string | `all` | Starting template for the environment:<br>`all` (clone full parent env), `core` (`HOME`, `PATH`, `USER`, …), or `none` (start empty). |
394-
| `ignore_default_excludes` | boolean | `false` | When `false`, Codex removes any var whose **name** contains `KEY`, `SECRET`, or `TOKEN` (case-insensitive) before other rules run. |
390+
| Field | Type | Default | Description |
391+
| ------------------------- | -------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
392+
| `inherit` | string | `all` | Starting template for the environment:<br>`all` (clone full parent env), `core` (`HOME`, `PATH`, `USER`, …), or `none` (start empty). |
393+
| `ignore_default_excludes` | boolean | `false` | When `false`, Codex removes any var whose **name** contains `KEY`, `SECRET`, or `TOKEN` (case-insensitive) before other rules run. |
395394
| `exclude` | array<string> | `[]` | Case-insensitive glob patterns to drop after the default filter.<br>Examples: `"AWS_*"`, `"AZURE_*"`. |
396395
| `set` | table<string,string> | `{}` | Explicit key/value overrides or additions – always win over inherited values. |
397396
| `include_only` | array<string> | `[]` | If non-empty, a whitelist of patterns; only variables that match _one_ pattern survive the final step. (Generally used with `inherit = "all"`.) |
@@ -566,60 +565,62 @@ Options that are specific to the TUI.
566565

567566
## Config reference
568567

569-
| Key | Type / Values | Notes |
570-
| --- | --- | --- |
571-
| `model` | string | Model to use (e.g., `gpt-5`). |
572-
| `model_provider` | string | Provider id from `model_providers` (default: `openai`). |
573-
| `model_context_window` | number | Context window tokens. |
574-
| `model_max_output_tokens` | number | Max output tokens. |
575-
| `approval_policy` | `untrusted` | `on-failure` | `on-request` | `never` | When to prompt for approval. |
576-
| `sandbox_mode` | `read-only` | `workspace-write` | `danger-full-access` | OS sandbox policy. |
577-
| `sandbox_workspace_write.writable_roots` | array<string> | Extra writable roots in workspace‑write. |
578-
| `sandbox_workspace_write.network_access` | boolean | Allow network in workspace‑write (default: false). |
579-
| `sandbox_workspace_write.exclude_tmpdir_env_var` | boolean | Exclude `$TMPDIR` from writable roots (default: false). |
580-
| `sandbox_workspace_write.exclude_slash_tmp` | boolean | Exclude `/tmp` from writable roots (default: false). |
581-
| `disable_response_storage` | boolean | Required for ZDR orgs. |
582-
| `notify` | array<string> | External program for notifications. |
583-
| `instructions` | string | Currently ignored; use `experimental_instructions_file` or `AGENTS.md`. |
584-
| `mcp_servers.<id>.command` | string | MCP server launcher command. |
585-
| `mcp_servers.<id>.args` | array<string> | MCP server args. |
586-
| `mcp_servers.<id>.env` | map<string,string> | MCP server env vars. |
587-
| `model_providers.<id>.name` | string | Display name. |
588-
| `model_providers.<id>.base_url` | string | API base URL. |
589-
| `model_providers.<id>.env_key` | string | Env var for API key. |
590-
| `model_providers.<id>.wire_api` | `chat` | `responses` | Protocol used (default: `chat`). |
591-
| `model_providers.<id>.query_params` | map<string,string> | Extra query params (e.g., Azure `api-version`). |
592-
| `model_providers.<id>.http_headers` | map<string,string> | Additional static headers. |
593-
| `model_providers.<id>.env_http_headers` | map<string,string> | Headers sourced from env vars. |
594-
| `model_providers.<id>.request_max_retries` | number | Per‑provider HTTP retry count (default: 4). |
595-
| `model_providers.<id>.stream_max_retries` | number | SSE stream retry count (default: 5). |
596-
| `model_providers.<id>.stream_idle_timeout_ms` | number | SSE idle timeout (ms) (default: 300000). |
597-
| `project_doc_max_bytes` | number | Max bytes to read from `AGENTS.md`. |
598-
| `profile` | string | Active profile name. |
599-
| `profiles.<name>.*` | various | Profile‑scoped overrides of the same keys. |
600-
| `history.persistence` | `save-all` | `none` | History file persistence (default: `save-all`). |
601-
| `history.max_bytes` | number | Currently ignored (not enforced). |
602-
| `file_opener` | `vscode` | `vscode-insiders` | `windsurf` | `cursor` | `none` | URI scheme for clickable citations (default: `vscode`). |
603-
| `tui` | table | TUI‑specific options (reserved). |
604-
| `hide_agent_reasoning` | boolean | Hide model reasoning events. |
605-
| `show_raw_agent_reasoning` | boolean | Show raw reasoning (when available). |
606-
| `model_reasoning_effort` | `minimal` | `low` | `medium` | `high` | Responses API reasoning effort. |
607-
| `model_reasoning_summary` | `auto` | `concise` | `detailed` | `none` | Reasoning summaries. |
608-
| `model_verbosity` | `low` | `medium` | `high` | GPT‑5 text verbosity (Responses API). |
609-
| `model_supports_reasoning_summaries` | boolean | Force‑enable reasoning summaries. |
610-
| `chatgpt_base_url` | string | Base URL for ChatGPT auth flow. |
611-
| `experimental_resume` | string (path) | Resume JSONL path (internal/experimental). |
612-
| `experimental_instructions_file` | string (path) | Replace built‑in instructions (experimental). |
613-
| `experimental_use_exec_command_tool` | boolean | Use experimental exec command tool. |
614-
| `responses_originator_header_internal_override` | string | Override `originator` header value. |
615-
| `projects.<path>.trust_level` | string | Mark project/worktree as trusted (only `"trusted"` is recognized). |
616-
| `preferred_auth_method` | `chatgpt` | `apikey` | Select default auth method (default: `chatgpt`). |
617-
| `tools.web_search` | boolean | Enable web search tool (alias: `web_search_request`) (default: false). |
568+
| Key | Type / Values | Notes |
569+
| ------------------------------------------------ | ------------------ | ----------------------------------------------------------------------- | ------------------------------------------------ | ------------------------------------- | ------------------------------- | ------------------------------------------------------- |
570+
| `model` | string | Model to use (e.g., `gpt-5`). |
571+
| `model_provider` | string | Provider id from `model_providers` (default: `openai`). |
572+
| `model_context_window` | number | Context window tokens. |
573+
| `model_max_output_tokens` | number | Max output tokens. |
574+
| `approval_policy` | `untrusted` | `on-failure` | `on-request` | `never` | When to prompt for approval. |
575+
| `sandbox_mode` | `read-only` | `workspace-write` | `danger-full-access` | OS sandbox policy. |
576+
| `sandbox_workspace_write.writable_roots` | array<string> | Extra writable roots in workspace‑write. |
577+
| `sandbox_workspace_write.network_access` | boolean | Allow network in workspace‑write (default: false). |
578+
| `sandbox_workspace_write.exclude_tmpdir_env_var` | boolean | Exclude `$TMPDIR` from writable roots (default: false). |
579+
| `sandbox_workspace_write.exclude_slash_tmp` | boolean | Exclude `/tmp` from writable roots (default: false). |
580+
| `disable_response_storage` | boolean | Required for ZDR orgs. |
581+
| `notify` | array<string> | External program for notifications. |
582+
| `instructions` | string | Currently ignored; use `experimental_instructions_file` or `AGENTS.md`. |
583+
| `mcp_servers.<id>.command` | string | MCP server launcher command. |
584+
| `mcp_servers.<id>.args` | array<string> | MCP server args. |
585+
| `mcp_servers.<id>.env` | map<string,string> | MCP server env vars. |
586+
| `model_providers.<id>.name` | string | Display name. |
587+
| `model_providers.<id>.base_url` | string | API base URL. |
588+
| `model_providers.<id>.env_key` | string | Env var for API key. |
589+
| `model_providers.<id>.wire_api` | `chat` | `responses` | Protocol used (default: `chat`). |
590+
| `model_providers.<id>.query_params` | map<string,string> | Extra query params (e.g., Azure `api-version`). |
591+
| `model_providers.<id>.http_headers` | map<string,string> | Additional static headers. |
592+
| `model_providers.<id>.env_http_headers` | map<string,string> | Headers sourced from env vars. |
593+
| `model_providers.<id>.request_max_retries` | number | Per‑provider HTTP retry count (default: 4). |
594+
| `model_providers.<id>.stream_max_retries` | number | SSE stream retry count (default: 5). |
595+
| `model_providers.<id>.stream_idle_timeout_ms` | number | SSE idle timeout (ms) (default: 300000). |
596+
| `project_doc_max_bytes` | number | Max bytes to read from `AGENTS.md`. |
597+
| `profile` | string | Active profile name. |
598+
| `profiles.<name>.*` | various | Profile‑scoped overrides of the same keys. |
599+
| `history.persistence` | `save-all` | `none` | History file persistence (default: `save-all`). |
600+
| `history.max_bytes` | number | Currently ignored (not enforced). |
601+
| `file_opener` | `vscode` | `vscode-insiders` | `windsurf` | `cursor` | `none` | URI scheme for clickable citations (default: `vscode`). |
602+
| `tui` | table | TUI‑specific options (reserved). |
603+
| `hide_agent_reasoning` | boolean | Hide model reasoning events. |
604+
| `show_raw_agent_reasoning` | boolean | Show raw reasoning (when available). |
605+
| `model_reasoning_effort` | `minimal` | `low` | `medium` | `high` | Responses API reasoning effort. |
606+
| `model_reasoning_summary` | `auto` | `concise` | `detailed` | `none` | Reasoning summaries. |
607+
| `model_verbosity` | `low` | `medium` | `high` | GPT‑5 text verbosity (Responses API). |
608+
| `model_supports_reasoning_summaries` | boolean | Force‑enable reasoning summaries. |
609+
| `chatgpt_base_url` | string | Base URL for ChatGPT auth flow. |
610+
| `experimental_resume` | string (path) | Resume JSONL path (internal/experimental). |
611+
| `experimental_instructions_file` | string (path) | Replace built‑in instructions (experimental). |
612+
| `experimental_use_exec_command_tool` | boolean | Use experimental exec command tool. |
613+
| `responses_originator_header_internal_override` | string | Override `originator` header value. |
614+
| `projects.<path>.trust_level` | string | Mark project/worktree as trusted (only `"trusted"` is recognized). |
615+
| `preferred_auth_method` | `chatgpt` | `apikey` | Select default auth method (default: `chatgpt`). |
616+
| `tools.web_search` | boolean | Enable web search tool (alias: `web_search_request`) (default: false). |
618617

619618
## Per-repo memory (local)
620619

621620
Codex records a lightweight, per-repository memory of key actions to help you and external tools recall decisions and changes made in a project. This data is written locally to your repo and never leaves your machine.
622621

622+
For backend options and maintenance commands, see [memory backends](./memory-backends.md).
623+
623624
- Location: `<repo>/.codex/memory/memory.jsonl` (one JSON object per line)
624625
- Scope: Automatically associated with the repository that contains `.git/` or `.codex/`.
625626
- When entries are added:
@@ -635,11 +636,13 @@ Example (one line):
635636
```
636637

637638
Notes
639+
638640
- This store is local-only and intended for project memory/history.
639641
- Clearing: delete or truncate the file at `<repo>/.codex/memory/memory.jsonl`.
640642
- Backups/exports: copy the JSONL file anywhere (each line is an entry).
641643

642644
Toggle
645+
643646
- Per run (both TUI and exec): pass `--memory off` to disable or `--memory on` to force-enable.
644647
- Environment variable (both modes): set `CODEX_PER_REPO_MEMORY=0|1` (also accepts `on|off`, `true|false`). Alias: `CODEX_MEMORY`.
645648
- Examples:

‎docs/memory-backends.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Memory backends
2+
3+
Codex stores per-repo state so you and the CLI can recall decisions across sessions. Two storage backends are available.
4+
5+
## JSONL (default)
6+
7+
- One JSON object per line, easy to inspect and back up.
8+
- Paths: `<repo>/.codex/memory/memory.jsonl` or `~/.codex/memory/memory.jsonl`.
9+
- Works out of the box and is diff‑friendly for version control.
10+
11+
## SQLite (optional)
12+
13+
- Adds atomic updates and indexes for faster queries.
14+
- Paths: `<repo>/.codex/memory/memory.db` or `~/.codex/memory/memory.db`.
15+
- Requires a build with the SQLite feature (`--features codex-memory/sqlite`).
16+
- Select at runtime with `CODEX_MEMORY_BACKEND=sqlite` (defaults to `jsonl`).
17+
18+
## Migrating existing data
19+
20+
Convert an existing JSONL store to SQLite:
21+
22+
```bash
23+
codex memory migrate
24+
```
25+
26+
After migration, enable SQLite with `CODEX_MEMORY_BACKEND=sqlite`.
27+
28+
## Compacting
29+
30+
Reclaim space and keep the store tidy:
31+
32+
```bash
33+
codex memory compact
34+
```
35+
36+
This vacuums a SQLite database or rewrites a JSONL file to drop unused entries.

0 commit comments

Comments
 (0)