perf(install_state): replace per-tool .mise.backend files with single index #7341

jdx · 2025-12-16T21:00:47Z

Summary

Replace individual .mise.backend files per installed tool with a single .mise.meta.toml index file
Reduces startup I/O from N file reads to 1 file read
Auto-migrates from legacy format on first run
Auto-cleans stale entries when tool directories are deleted

Index File Format

# ~/.local/share/mise/installs/.mise.meta.toml
[tools]
node = "core:node"
prettier = "npm:prettier"
eza = "cargo:eza[bin=eza]"

Test plan

Unit tests pass (377 tests)
E2E tests
Manual test: install a tool, verify index updated
Manual test: uninstall tool, verify entry removed from index
Manual test: startup with many tools is faster

🤖 Generated with Claude Code

Note

Consolidates per-tool .mise.backend files into a single atomic, locked .mise.meta.toml index and updates tool init and lookups to use it, plus adjusts symlink cleanup.

Backend metadata indexing:
- Introduce single index ~/.local/share/mise/installs/.mise.meta.toml with read_index/write_index and TOML schema (BackendIndex.tools).
- Auto-migrate from legacy per-tool .mise.backend files and prune entries for missing tool dirs.
- Atomic writes (temp + rename) and file locking in write_backend_meta to avoid races.
Tool initialization:
- init_tools reads the index once (removes per-tool file reads) and builds InstallStateTools from it.
- Lookups use kebab-cased keys in get_tool_full, backend_type, and list_versions.
Runtime symlinks cleanup:
- remove_missing_symlinks now ignores metadata files .mise.backend (legacy) and .mise.meta.toml when pruning empty install dirs.

^{Written by Cursor Bugbot for commit dc3407c. This will update automatically on new commits. Configure here.}

When config explicitly provides tool options, start with registry defaults only instead of inheriting cached options from .mise.backend. This prevents stale cached options (e.g., old 'url' option) from conflicting with new config options (e.g., platform-specific URLs). Fixes #7034 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

Instead of discarding all cached options when config provides new options, selectively filter out only install-time-only options while preserving post-install options like bin_path. Each backend now implements install_time_option_keys() to declare which options only affect installation/download and should not be inherited from cached .mise.backend when config provides its own options. - http: url, checksum, version_list_url, version_regex, version_json_path, format - github/gitlab: asset_pattern, url, version_prefix - ubi: exe, matching, matching_regex, provider - cargo: features, default-features, bin - go: tags - pipx: extras, pipx_args, uvx_args, uvx This allows: - Changing url/asset_pattern/checksum config without reinstall issues - Preserving post-install options like bin_path for binary discovery Fixes #7034 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

… index Replace individual `.mise.backend` files for each installed tool with a single `.mise.meta.toml` index file. This reduces startup I/O from N file reads (one per installed tool) to a single file read. Changes: - New index file at `~/.local/share/mise/installs/.mise.meta.toml` - TOML format: `[tools]` section with `short = "full"` entries - Auto-migration from legacy `.mise.backend` files on first run - Auto-cleanup of stale entries when tool directories are deleted - Atomic writes using temp file + rename pattern 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

…ndex

Copilot

Pull request overview

This PR optimizes startup performance by replacing per-tool .mise.backend files with a single .mise.meta.toml index file. This reduces I/O operations from N file reads (one per installed tool) to a single file read. The implementation includes automatic migration from the legacy format and cleanup of stale entries when tool directories are deleted.

Key changes:

Introduces centralized backend metadata index at ~/.local/share/mise/installs/.mise.meta.toml
Implements automatic migration from legacy .mise.backend files with cleanup after reading
Adds install-time-only option filtering to prevent conflicts when config options override cached metadata

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/toolset/install_state.rs	Core implementation of index-based backend metadata system with migration logic
src/runtime_symlinks.rs	Updates directory cleanup to include new `.mise.meta.toml` file
src/config/config_file/mise_toml.rs	Implements filtering of install-time-only options when merging config options
src/backend/ubi.rs	Defines install-time-only option keys for UBI backend
src/backend/pipx.rs	Defines install-time-only option keys for PIPX backend
src/backend/mod.rs	Adds dispatcher function for install-time-only option keys by backend type
src/backend/http.rs	Defines install-time-only option keys for HTTP backend
src/backend/go.rs	Defines install-time-only option keys for Go backend
src/backend/github.rs	Defines install-time-only option keys for GitHub/GitLab backend
src/backend/cargo.rs	Defines install-time-only option keys for Cargo backend

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-16T21:02:17Z

src/toolset/install_state.rs

+    let index = read_index();
    let mut jset = JoinSet::new();
    for dir in file::dir_subdirs(&dirs::INSTALLS)? {
+        let index = index.clone();


Cloning the entire index (BTreeMap) for each spawned task is inefficient. Since the index is only read in these tasks, wrap it in Arc before the loop to share a single allocation across all tasks instead of creating N copies.

Copilot · 2025-12-16T21:02:17Z

src/toolset/install_state.rs

+    let original_len = index.len();
+    index.retain(|short, _| {
+        let tool_dir = dirs::INSTALLS.join(short.to_kebab_case());
+        tool_dir.exists()
+    });
+
+    // Write back if we removed any stale entries
+    if index.len() != original_len {


The cleanup logic uses index.len() comparison to detect changes, which requires iterating the map twice (once for retain, once for len). Consider using a boolean flag set during retain or checking if any tools were removed to avoid the redundant length comparison.

Suggested change

let original_len = index.len();

index.retain(|short, _| {

let tool_dir = dirs::INSTALLS.join(short.to_kebab_case());

tool_dir.exists()

});

// Write back if we removed any stale entries

if index.len() != original_len {

let mut removed_any = false;

index.retain(|short, _| {

let tool_dir = dirs::INSTALLS.join(short.to_kebab_case());

let exists = tool_dir.exists();

if !exists {

removed_any = true;

}

exists

});

// Write back if we removed any stale entries

if removed_any {

Copilot · 2025-12-16T21:02:18Z

src/toolset/install_state.rs

+    // Clean up entries for tools whose directories no longer exist
+    let original_len = index.len();
+    index.retain(|short, _| {
+        let tool_dir = dirs::INSTALLS.join(short.to_kebab_case());


The kebab-case conversion is inconsistent with the index key usage. In line 216, the code looks up using dir directly, but cleanup converts keys with to_kebab_case(). If index keys are already in kebab-case, this conversion is redundant; if not, the lookup at line 216 may fail. Ensure consistent key formatting throughout the index operations.

Suggested change

// Clean up entries for tools whose directories no longer exist

let original_len = index.len();

index.retain(|short, _| {

let tool_dir = dirs::INSTALLS.join(short.to_kebab_case());

// Normalize all keys to kebab-case

let index: BTreeMap<String, String> = index

.into_iter()

.map(|(k, v)| (k.to_kebab_case(), v))

.collect();

// Clean up entries for tools whose directories no longer exist

let original_len = index.len();

let mut index = index;

index.retain(|short, _| {

let tool_dir = dirs::INSTALLS.join(short);

Copilot · 2025-12-16T21:02:18Z

src/toolset/install_state.rs

+            })
+            .unwrap_or_default();
+        let lines: Vec<&str> = body.lines().filter(|f| !f.is_empty()).collect();
+        let short = lines.first().unwrap_or(&dir).to_string();


Using unwrap_or with a reference to dir then converting to String is inefficient. Consider using lines.first().map(|s| s.to_string()).unwrap_or_else(|| dir.to_string()) to avoid the unnecessary reference and conversion.

Suggested change

let short = lines.first().unwrap_or(&dir).to_string();

let short = lines.first().map(|s| s.to_string()).unwrap_or_else(|| dir.to_string());

The .mise.backend.json format was deprecated long ago. Remove migration support to simplify the codebase. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

Get directory listing once with file::dir_subdirs() and use HashSet lookup instead of calling .exists() for each tool when cleaning up stale index entries. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

src/toolset/install_state.rs

The index file uses kebab-cased directory names as keys, but lookups were using the original short name (e.g., "gitlab:jdxcode/mise-test-fixtures"). This caused tools to not be found after installation. - write_backend_meta now uses ba.short.to_kebab_case() as the key - get_tool_full, backend_type, and list_versions now convert short to kebab-case before lookup - migrate_to_index uses the directory name (already kebab-cased) as key 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

During parallel tool installs, multiple processes could read the same index state, each insert their entry, and the second write would overwrite the first, losing data. Use LockFile to hold an exclusive lock during the entire read-modify-write operation to prevent concurrent modifications. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

cursor · 2025-12-16T21:33:44Z

src/toolset/install_state.rs

+        // Delete the legacy file after reading
+        if let Err(err) = file::remove_file(&path) {
+            debug!("Failed to remove legacy .mise.backend file: {err}");
+        }


Bug: Migration deletes legacy files before confirming write success

The read_legacy_backend_meta function deletes legacy .mise.backend files immediately after reading them (via file::remove_file), before migrate_to_index calls write_index. If the index write fails (e.g., disk full, permissions), the legacy files are already deleted and all backend metadata is permanently lost. The deletion should only occur after confirming the new index file was successfully written.

Additional Locations (1)

src/toolset/install_state.rs#L102-L108

cursor · 2025-12-16T21:33:45Z

src/toolset/install_state.rs

+        .ok()
+        .and_then(|content| toml::from_str::<BackendIndex>(&content).ok())
+        .map(|parsed| parsed.tools.into_iter().collect())
+        .unwrap_or_else(migrate_to_index);


Bug: Corrupted index file causes irreversible data loss

When the index file exists but contains invalid TOML (corruption), read_index falls back to migrate_to_index. However, after a successful prior migration, the legacy .mise.backend files were deleted. The migration will find no legacy files and return an empty map, causing all backend metadata to be silently lost. The code silently swallows the parse error with .ok() instead of distinguishing between "file missing" (migrate) and "file corrupted" (error).

cursor · 2025-12-16T21:33:45Z

src/toolset/install_state.rs

+
+    if index.len() != original_len {
+        let _ = write_index(&index);
+    }


Bug: Cleanup write in read_index races with write_backend_meta

The read_index function performs an unlocked write during stale entry cleanup (line 73), but write_backend_meta acquires a lock and also calls read_index. If init_tools calls read_index concurrently with write_backend_meta, the cleanup write could overwrite entries that were just added by write_backend_meta, causing newly installed tool metadata to be lost.

github-actions · 2025-12-16T21:57:38Z

Hyperfine Performance

`mise x -- echo`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.12.9 x -- echo`	20.9 ± 0.6	19.6	23.8	1.00
`mise x -- echo`	21.3 ± 0.7	20.0	24.4	1.02 ± 0.04

`mise env`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.12.9 env`	20.8 ± 0.9	19.2	26.6	1.01 ± 0.06
`mise env`	20.6 ± 0.7	19.2	23.7	1.00

`mise hook-env`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.12.9 hook-env`	20.9 ± 0.7	19.4	23.5	1.00
`mise hook-env`	21.8 ± 1.2	19.6	26.6	1.04 ± 0.07

`mise ls`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.12.9 ls`	17.8 ± 0.8	16.1	20.7	1.00
`mise ls`	20.8 ± 0.9	18.6	24.3	1.17 ± 0.07
⚠️ Warning: Performance variance for `ls` is 17%

`xtasks/test/perf`

Command	mise-2025.12.9	mise	Variance
install (cached)	108ms	⚠️ 124ms	-12%
ls (cached)	67ms	⚠️ 77ms	-12%
bin-paths (cached)	76ms	83ms	-8%
task-ls (cached)	298ms	⚠️ 2285ms	-86%

⚠️ Warning: install cached performance variance is -12%
⚠️ Warning: ls cached performance variance is -12%
⚠️ Warning: task-ls cached performance variance is -86%

jdx and others added 3 commits December 16, 2025 07:42

Copilot AI review requested due to automatic review settings December 16, 2025 21:00

Merge remote-tracking branch 'origin/main' into perf/single-backend-i…

d9b7924

…ndex

Copilot AI reviewed Dec 16, 2025

View reviewed changes

jdx and others added 3 commits December 16, 2025 15:04

refactor: remove support for legacy .mise.backend.json files

d446192

The .mise.backend.json format was deprecated long ago. Remove migration support to simplify the codebase. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

refactor: simplify read_index with and_then chain

78860ee

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

cursor bot reviewed Dec 16, 2025

View reviewed changes

src/toolset/install_state.rs Show resolved Hide resolved

src/toolset/install_state.rs Show resolved Hide resolved

jdx force-pushed the perf/single-backend-index branch from 7852f7b to d04b840 Compare December 16, 2025 21:25

cursor bot reviewed Dec 16, 2025

View reviewed changes

autofix-ci bot added 2 commits December 16, 2025 21:35

[autofix.ci] apply automated fixes

b108114

[autofix.ci] apply automated fixes (attempt 2/3)

71cd051

jdx marked this pull request as draft December 17, 2025 22:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

perf(install_state): replace per-tool .mise.backend files with single index #7341

perf(install_state): replace per-tool .mise.backend files with single index #7341

jdx commented Dec 16, 2025 •

edited by cursor bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 16, 2025

Uh oh!

Copilot AI Dec 16, 2025

Uh oh!

Copilot AI Dec 16, 2025

Uh oh!

Copilot AI Dec 16, 2025

Uh oh!

Uh oh!

Uh oh!

cursor bot Dec 16, 2025

Uh oh!

cursor bot Dec 16, 2025

Uh oh!

cursor bot Dec 16, 2025

Uh oh!

github-actions bot commented Dec 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	let short = lines.first().unwrap_or(&dir).to_string();
	let short = lines.first().map(\|s\| s.to_string()).unwrap_or_else(\|\| dir.to_string());

Uh oh!

perf(install_state): replace per-tool .mise.backend files with single index #7341

Are you sure you want to change the base?

perf(install_state): replace per-tool .mise.backend files with single index #7341

Conversation

jdx commented Dec 16, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Index File Format

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor bot Dec 16, 2025

Choose a reason for hiding this comment

Bug: Migration deletes legacy files before confirming write success

Uh oh!

cursor bot Dec 16, 2025

Choose a reason for hiding this comment

Bug: Corrupted index file causes irreversible data loss

Uh oh!

cursor bot Dec 16, 2025

Choose a reason for hiding this comment

Bug: Cleanup write in read_index races with write_backend_meta

Uh oh!

github-actions bot commented Dec 16, 2025

Hyperfine Performance

mise x -- echo

mise env

mise hook-env

mise ls

xtasks/test/perf

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jdx commented Dec 16, 2025 •

edited by cursor bot

Loading

`mise x -- echo`

`mise env`

`mise hook-env`

`mise ls`

`xtasks/test/perf`