Skip to content

[Pyomo.DoE] Add simultaneous design of multiple experiments#3866

Open
smondal13 wants to merge 188 commits into
Pyomo:mainfrom
smondal13:add-multiexperiment
Open

[Pyomo.DoE] Add simultaneous design of multiple experiments#3866
smondal13 wants to merge 188 commits into
Pyomo:mainfrom
smondal13:add-multiexperiment

Conversation

@smondal13
Copy link
Copy Markdown
Contributor

@smondal13 smondal13 commented Mar 2, 2026

Fixes # .

Summary/Motivation:

This PR adds a new DesignOfExperiments.optimize_experiments() API in pyomo/contrib/doe/doe.py to support simultaneous optimization of multiple experiments in one workflow. The motivation is to provide a multi-experiment DoE interface with stronger initialization options, clearer mode handling (template vs. user-initialized experiments), and richer diagnostics/results than the existing single-experiment path.

Changes proposed in this PR:

  • Added API optimize_experiments() for multi-experiment DoE optimization.
  • Added Cholesky-based D- and A-optimality objective and Greybox-based D-, A-, E-, and ME- optimality objective
  • Implemented two operating modes:
    • Template mode: pass one experiment and set n_exp.
    • User-initialized mode: pass a list of experiments; n_exp is inferred/validated.
  • Added optional LHS-based initialization (initialization_method="lhs") with controls for:
    • sample count, seed, candidate evaluation parallelism, combination fim metric scoring parallelism,
    • worker count, chunk size, parallel threshold, and optional wall-clock budget.
  • Added symmetry-breaking constraints for multi-experiment solves:
    • supports user-specified variable through sym_break_cons suffix,
    • falls back to the first experiment input with a diagnostic warning when not provided.
  • Expanded output for this API:
    • per-scenario and per-experiment results (designs, outputs, measurement errors, FIM/sensitivities),
    • aggregated FIM metrics, timings, settings, names, diagnostics, and structured run_info.
  • Added JSON-safe serialization via _DoEResultsJSONEncoder for numpy/Pyomo-enum values when writing results_file.

Note:

  • Added new documentation.md which describes the API. This documentation is to help the reviewers to understand the API and will not be merged into Pyomo:main

Remove before merging

  • documentation.md
  • rb_multi.py

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

…d replace `self.experiment` with `self.experiment_list[0]`. `doe/reactor_example.py` runs successfully.
…sing the `doe/reactor_multi_experiment.py`
…nt for `optimize_experiments()` and `compute_FIM()`
… both of sensitivity and optimize_experiments()
…A-opt gave different result for grid and optimization
Comment thread pyomo/contrib/doe/tests/test_greybox.py Outdated
@smondal13
Copy link
Copy Markdown
Contributor Author

@blnicho @mrmundt All the tests are passing except win/3.11 with

INFO: The following extensions were downloaded:
    [FAIL]  gjh
    [ OK ]  mcpp

Comment thread pyomo/contrib/doe/tests/test_doe_errors.py Outdated
smondal13 and others added 2 commits May 21, 2026 11:24
Co-authored-by: Miranda Mundt <55767766+mrmundt@users.noreply.github.com>
Comment thread pyomo/contrib/doe/tests/test_greybox.py Outdated
@smondal13
Copy link
Copy Markdown
Contributor Author

@mrmundt and @blnicho , all the comments are addressed, and all the tests are passing now.

@slilonfe5
Copy link
Copy Markdown
Member

@mrmundt @blnicho, Shuvo has addressed all the comments. This looks pretty good to me. Any final review would be appreciated; otherwise, I think we are good to go.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 27, 2026

Codecov Report

❌ Patch coverage is 91.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.15%. Comparing base (3603b1e) to head (a20061b).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
pyomo/contrib/doe/grey_box_utilities.py 91.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3866      +/-   ##
==========================================
+ Coverage   90.11%   90.15%   +0.03%     
==========================================
  Files         909      909              
  Lines      108580   109236     +656     
==========================================
+ Hits        97847    98477     +630     
- Misses      10733    10759      +26     
Flag Coverage Δ
builders 28.97% <91.66%> (-0.13%) ⬇️
default 86.17% <91.66%> (?)
expensive 34.97% <91.66%> (?)
linux 87.68% <91.66%> (-1.93%) ⬇️
linux_other 87.68% <91.66%> (+0.05%) ⬆️
oldsolvers 27.90% <8.33%> (-0.15%) ⬇️
osx 83.09% <8.33%> (+0.03%) ⬆️
win 85.44% <91.66%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@adowling2
Copy link
Copy Markdown
Member

@smondal13 This PR is now showing merge conflicts.

Copy link
Copy Markdown
Member

@adowling2 adowling2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is practically ready. I requested a few small comments. Also, there is a merge conflict (I think).

Comment thread pyomo/contrib/doe/tests/test_doe_build.py Outdated
Comment thread pyomo/contrib/doe/tests/test_doe_build.py
Comment thread pyomo/contrib/doe/tests/test_doe_build.py
@smondal13
Copy link
Copy Markdown
Contributor Author

smondal13 commented May 28, 2026

I think this is practically ready. I requested a few small comments. Also, there is a merge conflict (I think).

@adowling2 I have addressed your comments. @adowling2 @mrmundt, I have resolved the conflicts. The conflicts were from @adowling2's recent PR #3867 merge into Pyomo:main.

@smondal13
Copy link
Copy Markdown
Contributor Author

@mrmundt @blnicho, the following two tests are failing. I think these two are not part of my PR

=========================== short test summary info ============================
FAILED pyomo/contrib/iis/tests/test_iis.py::TestIIS::test_write_iis_any_solver - SystemError: <method 'write' of '_io.FileIO' objects> returned a result with an exception set
FAILED pyomo/contrib/iis/tests/test_iis.py::TestIIS::test_write_iis_cplex - SystemError: <method 'write' of '_io.FileIO' objects> returned a result with an exception set

@blnicho
Copy link
Copy Markdown
Member

blnicho commented May 28, 2026

@smondal13 those failures were resolved in #3965 and I merged main into your PR to pick up the fix.

@smondal13
Copy link
Copy Markdown
Contributor Author

@blnicho @mrmundt this one is failing due to download issue

downloaded:
[FAIL] gjh
[ OK ] mcpp
Error: Process completed with exit code 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Ready for design review

Development

Successfully merging this pull request may close these issues.

7 participants