perf(pm): fast path registry cache package clones#2979
Conversation
elrrrrrrr
added
A-Pkg Manager
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces prefetching for registry tarballs and refactors the installation scheduler to utilize a synchronous worker pool for extraction and cloning operations. It adds synchronous utility functions for cache lookups and directory cloning to optimize performance on warm installs. Feedback was provided regarding the removal of a redundant HashSet in the directory creation logic to avoid unnecessary allocations.
| let mut created_dirs = HashSet::new(); | ||
| for dir in &dirs { | ||
| if created_dirs.insert(dir.clone()) | ||
| && let Err(e) = fs::create_dir_all(dir) | ||
| && e.kind() != io::ErrorKind::AlreadyExists | ||
| { | ||
| return Err(e).with_context(|| err_msg.clone()); | ||
| } | ||
| } |
There was a problem hiding this comment.
The created_dirs HashSet is redundant here. The dirs vector is populated by collect_entries using a depth-first tree traversal, which naturally visits each directory exactly once. Removing this set avoids unnecessary allocations and lookups in the hot path of package cloning.
for dir in &dirs {
if let Err(e) = fs::create_dir_all(dir)
&& e.kind() != io::ErrorKind::AlreadyExists
{
return Err(e).with_context(|| err_msg.clone());
}
}
📊 pm-bench-phases ·
|
1 participant
Summary
package/wrapper directlyHypothesis
Warm registry installs currently call the generic source-discovery path before every cache clone, which costs at least one directory scan per package. Since npm registry tarballs are extracted into
<cache>/<name>/<version>/package, the scheduler can skip that scan for registry download/cache-hit paths and reduce p4 ctx/syscall without changing non-registry cache semantics.Validation
cargo fmtCARGO_NET_OFFLINE=true cargo test -p utoo-pm util::cloner::tests::test_clone_registry_package_from_cache_sync_uses_package_dirCARGO_NET_OFFLINE=true cargo test -p utoo-pm service::install_scheduler::testsCARGO_NET_OFFLINE=true cargo clippy -p utoo-pm --all-targets -- -D warnings --no-depsBenchmark Result
GHA run: https://github.com/utooland/utoo/actions/runs/26025415891
npmjs phase bench:
Conclusion: this does not currently prove a ctx/syscall win. #2975 already had p4 around
13.1-13.7K / 9.3-9.9K; this branch is14.1K / 9.2K. Keep as rejected/hold unless repeated runs show otherwise.