perf(pm): inline cache hits with lockfile clone prefetch by elrrrrrrr · Pull Request #2978 · utooland/utoo

elrrrrrrr · 2026-05-18T08:17:09Z

Summary

combine perf(pm): inline warm registry cache hits #2975 inline warm registry cache hits with perf(pm): prefetch lockfile clones #2976 lockfile clone prefetch
keep the perf(pm): prefetch lockfile installs with rayon workers #2974 base combo: lockfile registry download prefetch + extract/clone workers on rayon
test whether the p3 wall gain from early clone queueing can coexist with the p4 ctx win from inline cache hits

Hypothesis

#2975 is currently the best balanced candidate: p3 5.69s / ctx 66.0K-37.9K and p4 2.17s / ctx 13.1K-9.3K. #2976 has faster p3 wall at 5.22s but regresses p4 ctx to 21.7K-13.6K. This branch checks whether inline cache hits remove the p4 overhead while preserving #2976 p3.

Validation

cargo fmt
cargo test -p utoo-pm service::install::tests::test_parent_package_path
cargo test -p utoo-pm service::install_scheduler::tests
cargo test -p utoo-pm util::downloader::tests
cargo clippy -p utoo-pm --all-targets -- -D warnings --no-deps

Benchmark

GHA linux/npmjs, run 26021739493:

phase	utoo wall	utoo ctx	#2975 wall	#2975 ctx	utoo-next wall	bun wall
p0_full_cold	7.60s ±0.13	83.7K / 50.2K	7.59s ±0.20	74.2K / 44.6K	8.50s ±0.58	9.26s ±0.30
p1_resolve	2.63s ±0.14	16.9K / 21.9K	2.32s ±0.04	15.3K / 19.8K	3.23s ±0.05	2.31s ±0.06
p3_cold_install	7.04s ±0.22	84.6K / 47.9K	5.69s ±0.18	66.0K / 37.9K	5.99s ±0.03	6.68s ±0.20
p4_warm_link	2.09s ±0.08	17.5K / 11.5K	2.17s ±0.07	13.1K / 9.3K	2.71s ±0.62	3.34s ±0.05

Conclusion: reject. The clone-prefetch part does not combine with inline cache-hit: p3 regresses badly and p4 ctx also loses the #2975 benefit. Keep #2975 as the main candidate; do not fold #2976/#2978 into the final stack.

gemini-code-assist

Code Review

This pull request implements prefetching for package downloads and clones while refactoring the installation scheduler to use synchronous I/O operations within a worker pool. It introduces several synchronous utility functions for cloning, downloading, and extraction to support this architecture. The review feedback identifies a violation of panic recovery principles, suggests extracting shared filtering logic to reduce duplication in prefetching functions, and recommends stricter error handling when preparing target directories for cloning.

gemini-code-assist · 2026-05-18T08:19:29Z

+                let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
+                    clone_package_from_cache_sync(
+                        &job.spec.package.name,
+                        &job.spec.package.version,
+                        &job.spec.package.tarball_url,
+                        &job.cache_path,
+                        &job.spec.target,
+                    )
+                    .map_err(|e| format!("{e:#}"))
+                }))
+                .unwrap_or_else(|_| Err("install clone worker panicked".to_string()));


The use of std::panic::catch_unwind here violates the general rule regarding panic recovery. Panics should be treated as unrecoverable bugs that indicate logic errors, rather than being caught and converted into error results to allow the process to continue. If a worker panics, it should be allowed to propagate to ensure the bug is surfaced and fixed.

let result = clone_package_from_cache_sync( &job.spec.package.name, &job.spec.package.version, &job.spec.package.tarball_url, &job.cache_path, &job.spec.target, ) .map_err(|e| format!("{e:#}"));

References

Do not implement recovery logic for panics. Panics should be treated as unrecoverable bugs that need to be fixed, not as transient, recoverable errors.

gemini-code-assist · 2026-05-18T08:19:29Z

+fn prefetch_lock_downloads(
+    groups: &HashMap<usize, Vec<(String, Package)>>,
+    omit: &HashSet<OmitType>,
+    scheduler: &super::install_scheduler::InstallScheduler,
+) {
+    let registry = get_registry();
+
+    for packages in groups.values() {
+        for (path, package) in packages {
+            if should_omit_package(package, omit) || package.link.is_some() {
+                continue;
+            }
+
+            let Some(resolved) = package.resolved.as_deref() else {
+                continue;
+            };
+            if !is_prefetchable_registry_tarball(resolved, &registry) {
+                continue;
+            }
+
+            if package
+                .cpu
+                .as_ref()
+                .is_some_and(|cpu| !is_cpu_compatible(cpu))
+                || package.os.as_ref().is_some_and(|os| !is_os_compatible(os))
+            {
+                continue;
+            }
+
+            let Some(version) = package.version.as_deref() else {
+                continue;
+            };
+            let name = package.get_name(path);
+            if name.is_empty() || name == "root" || name == "unknown" {
+                continue;
+            }
+
+            scheduler.prefetch_download(name, version.to_string(), resolved.to_string());
+        }
+    }
+}
+
+fn parent_package_path(path: &str) -> Option<&str> {
+    let index = path.rfind("/node_modules/")?;
+    Some(&path[..index])
+}
+
+fn prefetch_lock_clones(
+    groups: &HashMap<usize, Vec<(String, Package)>>,
+    cwd: &Path,
+    omit: &HashSet<OmitType>,
+    scheduler: &super::install_scheduler::InstallScheduler,
+) {
+    let registry = get_registry();
+    let mut depths: Vec<_> = groups.keys().copied().collect();
+    depths.sort_unstable();
+
+    for depth in depths {
+        let Some(packages) = groups.get(&depth) else {
+            continue;
+        };
+
+        for (path, package) in packages {
+            if should_omit_package(package, omit) || package.link.is_some() {
+                continue;
+            }
+
+            let Some(resolved) = package.resolved.as_deref() else {
+                continue;
+            };
+            if !is_prefetchable_registry_tarball(resolved, &registry) {
+                continue;
+            }
+
+            if package
+                .cpu
+                .as_ref()
+                .is_some_and(|cpu| !is_cpu_compatible(cpu))
+                || package.os.as_ref().is_some_and(|os| !is_os_compatible(os))
+            {
+                continue;
+            }
+
+            let Some(version) = package.version.as_deref() else {
+                continue;
+            };
+            let name = package.get_name(path);
+            if name.is_empty() || name == "root" || name == "unknown" {
+                continue;
+            }
+
+            scheduler.prefetch_clone(
+                name,
+                version.to_string(),
+                resolved.to_string(),
+                cwd.join(path),
+                parent_package_path(path).map(|parent| cwd.join(parent)),
+            );
+        }
+    }
+}


There is significant code duplication between prefetch_lock_downloads and prefetch_lock_clones regarding the logic for filtering eligible packages (checking omit status, links, compatibility, and valid names). Consider extracting this filtering logic into a shared helper function to improve maintainability and reduce the risk of logic drift between the two prefetch paths.

gemini-code-assist · 2026-05-18T08:19:29Z

+    if dst.try_exists()?
+        && let Err(e) = std::fs::remove_dir_all(dst)
+    {
+        tracing::warn!("Failed to clean target directory {}: {}", dst.display(), e);
+    }


Ignoring the error from remove_dir_all and only logging a warning is risky. If the directory cannot be cleaned (e.g., due to permission issues or open file handles), the subsequent clone operation will likely fail or result in a corrupted state. It is safer to propagate the error to ensure the environment is in a known clean state before cloning. Additionally, this removal logic is redundant with the check already performed in clone_package_sync.

if dst.try_exists()? { std::fs::remove_dir_all(dst).with_context(|| format!("Failed to clean target directory {}", dst.display()))?; }

github-actions · 2026-05-18T08:55:09Z

📊 pm-bench-phases · `bb3b93b` · linux (`ubuntu-latest`)

Workflow run — ant-design

PMs: utoo (this branch) · utoo-npm (latest published) · bun (latest)

npmjs.org

p0_full_cold

PM	wall	±σ	user	sys	RSS	pgMinor
bun	9.21s	0.27s	10.45s	10.36s	792M	340.1K
utoo-next	8.16s	0.18s	10.38s	12.19s	971M	121.8K
utoo-npm	8.89s	0.91s	10.85s	12.51s	1015M	133.6K
utoo	8.40s	1.26s	11.18s	11.99s	873M	137.7K

PM	vCtx	iCtx	netRX	netTX	cache	node_mod	lock
bun	15.2K	18.8K	1.20G	6M	1.90G	1.77G	1M
utoo-next	119.1K	77.9K	1.17G	5M	1.73G	1.72G	2M
utoo-npm	135.7K	91.6K	1.17G	5M	1.73G	1.72G	2M
utoo	92.9K	48.5K	1.18G	6M	1.72G	1.72G	2M

p1_resolve

PM	wall	±σ	user	sys	RSS	pgMinor
bun	1.94s	0.07s	4.10s	1.06s	499M	162.7K
utoo-next	3.08s	0.09s	5.53s	2.05s	619M	90.0K
utoo-npm	3.01s	0.06s	5.52s	2.06s	613M	84.4K
utoo	2.32s	0.02s	6.03s	1.63s	643M	117.1K

PM	vCtx	iCtx	netRX	netTX	cache	node_mod	lock
bun	8.0K	4.9K	204M	3M	108M	-	1M
utoo-next	72.1K	88.0K	201M	2M	7M	3M	2M
utoo-npm	71.9K	90.5K	201M	2M	7M	3M	2M
utoo	13.5K	19.4K	204M	3M	7M	3M	2M

p3_cold_install

PM	wall	±σ	user	sys	RSS	pgMinor
bun	6.96s	0.19s	6.34s	9.97s	643M	212.4K
utoo-next	5.93s	0.11s	4.98s	10.85s	475M	62.8K
utoo-npm	6.34s	0.43s	5.10s	11.06s	522M	68.8K
utoo	5.56s	0.28s	4.76s	10.46s	468M	55.9K

PM	vCtx	iCtx	netRX	netTX	cache	node_mod	lock
bun	4.5K	6.9K	1.00G	3M	1.77G	1.77G	1M
utoo-next	100.5K	44.4K	998M	3M	1.72G	1.72G	2M
utoo-npm	105.3K	44.2K	999M	3M	1.72G	1.72G	2M
utoo	65.4K	37.1K	998M	3M	1.72G	1.72G	2M

p4_warm_link

PM	wall	±σ	user	sys	RSS	pgMinor
bun	3.37s	0.06s	0.21s	2.37s	134M	32.6K
utoo-next	2.62s	0.19s	0.51s	3.83s	80M	19.0K
utoo-npm	2.43s	0.02s	0.51s	3.81s	78M	18.1K
utoo	2.28s	0.02s	0.41s	3.52s	54M	11.9K

PM	vCtx	iCtx	netRX	netTX	cache	node_mod	lock
bun	536	26	5M	33K	1.93G	1.76G	1M
utoo-next	41.6K	18.3K	6K	5K	1.72G	1.72G	2M
utoo-npm	39.0K	17.4K	7K	10K	1.72G	1.72G	2M
utoo	17.2K	10.8K	6K	20K	1.72G	1.72G	2M

npmmirror.com: no output captured.

elrrrrrrr · 2026-05-20T04:00:22Z

Closing this PM performance experiment after the investigation phase. The benchmark data and conclusions are preserved in the PR body/comments; we will split the validated pieces into smaller reviewable PRs for the formal ship path.

elrrrrrrr added 6 commits May 18, 2026 15:03

perf(pm): run install clone workers on rayon

5ae8db1

perf(pm): skip seeded probe for registry installs

49110c1

perf(pm): prefetch lockfile registry downloads

90e219f

perf(pm): run install extract workers on rayon

4f1cea8

perf(pm): inline warm registry cache hits

9ad334f

perf(pm): prefetch lockfile clones

636e368

elrrrrrrr added A-Pkg Manager Area: Package Manager benchmark Run pm-bench on PR labels May 18, 2026

gemini-code-assist Bot reviewed May 18, 2026

View reviewed changes

elrrrrrrr mentioned this pull request May 18, 2026

perf(pm): inline warm registry cache hits #2975

Closed

elrrrrrrr closed this May 20, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf(pm): inline cache hits with lockfile clone prefetch#2978

perf(pm): inline cache hits with lockfile clone prefetch#2978
elrrrrrrr wants to merge 6 commits into
perf/pm-resolver-demand-bfsfrom
exp/pm-install-inline-cache-prefetch-clones-b33d922

elrrrrrrr commented May 18, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot May 18, 2026

Uh oh!

gemini-code-assist Bot May 18, 2026

Uh oh!

gemini-code-assist Bot May 18, 2026

Uh oh!

github-actions Bot commented May 18, 2026

Uh oh!

elrrrrrrr commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

elrrrrrrr commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Hypothesis

Validation

Benchmark

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 18, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 18, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 18, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 18, 2026

📊 pm-bench-phases · bb3b93b · linux (ubuntu-latest)

npmjs.org

p0_full_cold

p1_resolve

p3_cold_install

p4_warm_link

Uh oh!

elrrrrrrr commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

elrrrrrrr commented May 18, 2026 •

edited

Loading

📊 pm-bench-phases · `bb3b93b` · linux (`ubuntu-latest`)