Skip to content

Add rough CPU memory requirements estimation functionality#722

Open
yousefmoazzam wants to merge 7 commits into
mainfrom
cpu-memory-check
Open

Add rough CPU memory requirements estimation functionality#722
yousefmoazzam wants to merge 7 commits into
mainfrom
cpu-memory-check

Conversation

@yousefmoazzam
Copy link
Copy Markdown
Collaborator

Fixes IMGDA-865

It's worth noting that at the moment, the command does take a few seconds to run:

(tomography) [twi18192@pc0074 twi18192]$ time python -m httomo memory-check /dls/i13/data/2026/cm44160-1/zihan_recon/raw/191449.nxs /dls/science/users/twi18192/httomo/tests/samples/pipeline_template_examples/paganin_lprec_crash.yaml 2
Estimated peak CPU memory usage for 2 process run: 514.791 GB

real    0m4.168s
user    0m2.627s
sys     0m0.320s

Profiling shows that the majority of the time taken is in the YAML file loading and parsing, as part of the Pipeline object generation. It's not clear how to go about optimising this as loading is IO bound and parsing is something we offload to an external library, so I have left any attempt to optimise this for later.

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have made corresponding changes to the documentation
  • I have added the user-release-note label in order to include this PR in the "Notable
    Changes for Users" section in release notes

@yousefmoazzam yousefmoazzam requested a review from dkazanc May 26, 2026 09:22
@yousefmoazzam yousefmoazzam added the user-release-note Label for a PR to be included in the "user release notes" section of release notes label May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

user-release-note Label for a PR to be included in the "user release notes" section of release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant