[Pyomo.DoE] Add simultaneous design of multiple experiments by smondal13 · Pull Request #3866 · Pyomo/pyomo

smondal13 · 2026-03-02T19:53:03Z

Fixes # .

Summary/Motivation:

This PR adds a new DesignOfExperiments.optimize_experiments() API in pyomo/contrib/doe/doe.py to support simultaneous optimization of multiple experiments in one workflow. The motivation is to provide a multi-experiment DoE interface with stronger initialization options, clearer mode handling (template vs. user-initialized experiments), and richer diagnostics/results than the existing single-experiment path.

Changes proposed in this PR:

Added API optimize_experiments() for multi-experiment DoE optimization.
Implemented two operating modes:
- Template mode: pass one experiment and set n_exp.
- User-initialized mode: pass a list of experiments; n_exp is inferred/validated.
Added optional LHS-based initialization (initialization_method="lhs") with controls for:
- sample count, seed, candidate evaluation parallelism, combination fim metric scoring parallelism,
- worker count, chunk size, parallel threshold, and optional wall-clock budget.
Added symmetry-breaking constraints for multi-experiment solves:
- supports user-specified variable through sym_break_cons suffix,
- falls back to the first experiment input with a diagnostic warning when not provided.
Expanded output for this API:
- per-scenario and per-experiment results (designs, outputs, measurement errors, FIM/sensitivities),
- aggregated FIM metrics, timings, settings, names, diagnostics, and structured run_info.
Added JSON-safe serialization via _DoEResultsJSONEncoder for numpy/Pyomo-enum values when writing results_file.

Note:

Added new documentation.md which describes the API. This documentation is to help the reviewers to understand the API and will not be merged into Pyomo:main

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the BSD license.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

…`experiment` argument is provided.

…d replace `self.experiment` with `self.experiment_list[0]`. `doe/reactor_example.py` runs successfully.

…sing the `doe/reactor_multi_experiment.py`

…ultiexperiment.py` both for 3 experiments

…etric-uncertainty

…parametric-uncertainty

…riment

…nt for `optimize_experiments()` and `compute_FIM()`

… both of sensitivity and optimize_experiments()

…A-opt gave different result for grid and optimization

smondal13 · 2026-03-03T13:54:20Z

pyomo/contrib/doe/doe.py

    def __init__(
        self,
        experiment=None,
+        experiment_list=None,


@adowling2 , @blnicho Should we keep both experiment and experiment_list? The user can pass the experiment_list argument if the user wants to pass the user-initialized experiment object. Another option is to keep the experiment argument and make it able to accept list objects as well, and maybe use a hint, e.g., experiment: list.

Currently, I have added a deprecation warning for the experiment argument.

I like allowing experiment to be either an instance of the Experiment class (or a child) or a list of Experiment objects. That seems cleaner to me.

pyomo/contrib/doe/doe.py

smondal13 · 2026-03-03T14:12:37Z

pyomo/contrib/doe/doe.py

+            "variable": None,
+            "source": None,
+        }
+        diagnostics_warnings = []


I want to propagate the warning to the final result, which may help the user. If this is not a good choice, let me know @adowling2, @blnicho

…tency

…ing and improve error handling; add utility methods for enum labels and input fixing. Run black

smondal13 · 2026-03-03T19:16:58Z

@blnicho @adowling2 This PR is ready for an initial review. Please take a look when you have a moment.

smondal13 · 2026-03-05T01:41:23Z

pyomo/contrib/doe/doe.py

+    lhs = "lhs"
+
+
+class _DoEResultsJSONEncoder(json.JSONEncoder):


Numpy floats specifically are shown in a weird format, which is why everything is converted to Python objects.

This looks strange to me. Are you saving DoE results to a JSON?

That is one of the options. The user can choose to save the result.

smondal13 · 2026-03-05T01:46:51Z

pyomo/contrib/doe/doe.py

+        init_n_workers: int = None,
+        init_combo_chunk_size: int = 5000,
+        init_combo_parallel_threshold: int = 20000,
+        init_max_wall_clock_time: float = None,


@adowling2, @blnicho, It may be worth adding an init_solver = None where the user can add flexible solver options that will be passed for initialization. It may save time for initialization. If the user does not pass that, then the default solver from DesignOfExperiments will be used. Let me know if this is a good idea.

I have been thinking something similar to this... it would be helpful to allow for different solver options: initialization and the full optimization solve.

smondal13 · 2026-03-05T01:51:29Z

pyomo/contrib/doe/doe.py

+        diagnostics_warnings = []
+
+        # Add experiment(s) for each scenario
+        # TODO: Add s_prev = 0 to handle parameter scenarios


This is the extension that I will be working on next. So, I have added this as a reference for myself. This is the reason we are using param_scenarios, although we are not using it this time. It does not interfere with the user interface.

smondal13 · 2026-03-06T17:43:41Z

@blnicho One of the tests is failing.

… add-multiexperiment

…for `_DoEResultsJSONEncoder` and `optimize_experiments()` API

- update LHS candidate FIM evaluation logging to report the actual\n execution mode so users can immediately see whether initialization\n is running serially or in parallel.\n- add an explicit warning when lhs_parallel=True resolves to\n serial execution because only one worker is available, preventing\n silent fallback confusion during debugging.\n- keep LHS initialization behavior unchanged while improving\n observability for multi-experiment startup diagnostics.

adowling2

There is a lot of added code in this PR. I think it would be helpful to draft a documentation page explaining the key features of this proposed interface. This would be a simple .md file that eventually becomes a RTD page. I find that drafting documentation helps provide more robust motivation and context to engage with the PR.

adowling2 · 2026-03-17T17:39:38Z

pyomo/contrib/doe/tests/_review.md

@@ -0,0 +1,362 @@
+# Review: `_DoEResultsJSONEncoder`, `_enum_label`, and `optimize_experiments()` API


How was this file generated? I am looking for some additional context to help me digest the contents of this file. @smondal13

This was for me to review. I forgot to remove this. I asked AI to review the code for edge cases and best practices.

adowling2 · 2026-03-17T17:40:59Z

pyomo/contrib/doe/tests/test_doe_build.py

    ]
-    sigma_inv = [1 / v for k, v in model.scenario_blocks[0].measurement_error.items()]
+    sigma_inv = [
+        1 / v for k, v in model.fd_scenario_blocks[0].measurement_error.items()


I like this!

adowling2 · 2026-03-17T17:42:31Z

pyomo/contrib/doe/tests/test_doe_build.py

+            objective_option="pseudo_trace",
+        )
+
+        model_direct = pyo.ConcreteModel()


What is this block of code doing? Comments would help.

This is used to check the build structure of the new method. This one particularly tests the helper function that returns the experiment_input vars

adowling2 · 2026-03-17T17:43:54Z

pyomo/contrib/doe/tests/test_doe_build.py

+
+    @unittest.skipIf(not ipopt_available, "The 'ipopt' command is not available")
+    def test_multi_experiment_structure_and_results(self):
+        solver = self._make_solver()


What is this test doing? I recommend adding a comment block with a few-sentence description of the contents and goals of this test.

This applies to all of the tests in this file.

adowling2 · 2026-03-17T17:45:11Z

pyomo/contrib/doe/doe.py

 import json
 import logging
 import math
+import os
+import threading


We want to discuss with the Pyomo team if this is the recommended approach for threading.

pyomo/contrib/doe/doe.py

adowling2 · 2026-03-17T17:49:50Z

pyomo/contrib/doe/doe.py

+        init_n_workers: int = None,
+        init_combo_chunk_size: int = 5000,
+        init_combo_parallel_threshold: int = 20000,
+        init_max_wall_clock_time: float = None,


I have been thinking something similar to this... it would be helpful to allow for different solver options: initialization and the full optimization solve.

adowling2 · 2026-03-17T17:52:00Z

pyomo/contrib/doe/doe.py

@@ -537,6 +619,1555 @@ def run_doe(self, model=None, results_file=None):
            with open(results_file, "w") as file:
                json.dump(self.results, file)

+    def optimize_experiments(


Is this function new? Does it replace an exisiting function?

This is a new helper function. Instead of writing this code multiple times, I have added a function that just do this.

- Updated the `DesignOfExperiments` class to accept either a single experiment or a list of experiments through the `experiment` parameter, replacing the previous `experiment_list` parameter. - Modified related test cases to reflect this change, ensuring that both single and multiple experiments are handled correctly. - Adjusted error messages and validation checks to align with the new parameter naming. - Ensured backward compatibility by updating all references in the test suite and error handling.

- Add documentation

…ntation

… add-multiexperiment

- enhance documentation for init_solver usage in multi-experiment optimization

smondal13 added 30 commits January 14, 2026 10:03

Add _spec.md

ed6707a

Add optimize_experiment()

eed6329

Add experiment_list in the __init__ and a DeprecationWarningif …

4861662

…`experiment` argument is provided.

Rename finite difference scenario_blocks to fd_scenario_blocks an…

bf5eb91

…d replace `self.experiment` with `self.experiment_list[0]`. `doe/reactor_example.py` runs successfully.

Add new objective method for multiple experiment.

f70e461

Add complete doe.py for multi experiment without testing /example

7c7332b

Solve the optimize_experiments() for both 1 and 2 experiment case u…

f5ba35e

…sing the `doe/reactor_multi_experiment.py`

Solve doe/examples/rooney_biegler_multiexperiment.py and `reactor_m…

79d09cc

…ultiexperiment.py` both for 3 experiments

Add plot and the grid best and optimal point points to the same point

fc0f8a5

Add minor changes

1a80a6c

Merge branch 'change-a-optimality' into add-multiexperiment-and-param…

948e77c

…etric-uncertainty

Add trace objective

b83ce4f

Merge branch 'add-comp-efficient-tests' into add-multiexperiment-and-…

63488f0

…parametric-uncertainty

Resolve the test failing issue for scenario_blocks does not exist.

6a53dc9

Correct docstring

4fa587c

Add trace and pseudo_trace in the plotting function for multiexpe…

422f0c2

…riment

A-optimality gives different results for rooney_biegler_multiexperime…

4bb38dc

…nt for `optimize_experiments()` and `compute_FIM()`

Add new directory for multiexperiment protype examples

974598b

Solve the error FIM not positive definite when prior_FIM is passed

c38875b

Solve rooney_biegler_multiexperiment.py

62f28ed

Add FIM prior in multiexperiment

42b8744

Add multiexperiment_prototype.ipynb that gives the same result from…

26ea0a6

… both of sensitivity and optimize_experiments()

When no prior was used, D-opt and trace FIM gave he same result, but …

cb19d00

…A-opt gave different result for grid and optimization

Solve 2 experiment case with same result from both grid and optimization

e56276a

Merge branch 'Pyomo:main' into add-multiexperiment

0f7cc9e

Add local_name of sym_break_cons

03bfb89

Add log10 codiition number

e9d33b7

Add lhs iniatialization method and test

0833295

Add tests for optimize_experiments(). All tests passed

ca0a4dd

Remove cherry-picked example artifacts from DOE test changes

02aa81a

smondal13 commented Mar 3, 2026

View reviewed changes

pyomo/contrib/doe/doe.py Outdated Show resolved Hide resolved

smondal13 commented Mar 3, 2026

View reviewed changes

smondal13 added 3 commits March 3, 2026 09:33

Update documentation and imports in DoE module for clarity and consis…

eef390a

…tency

Run black

280b01a

Refactor DesignOfExperiments to standardize initialization method nam…

c6be263

…ing and improve error handling; add utility methods for enum labels and input fixing. Run black

blnicho self-requested a review March 3, 2026 19:43

smondal13 commented Mar 5, 2026

View reviewed changes

Merge branch 'main' into add-multiexperiment

592d54c

smondal13 added 5 commits March 8, 2026 15:44

Fix multi-experiment compute_FIM aggregation

1f8f016

Merge branch 'add-multiexperiment' of github.com:smondal13/pyomo into…

ff5e465

… add-multiexperiment

Run black

967e790

Refactor _enum_label method placement and add review documentation …

8ee7b2b

…for `_DoEResultsJSONEncoder` and `optimize_experiments()` API

adowling2 reviewed Mar 17, 2026

View reviewed changes

smondal13 added 10 commits March 17, 2026 14:41

Remove the review file

c4103d9

Add comments to explain the test

1518abe

- update init_method type hint in DesignOfExperiments constructor

ee6e63f

- Add documentation

fix(doc): clarify design variables and FIM in multi-experiment docume…

ebc7aa3

…ntation

Rename documentation

8f4732d

Merge branch 'main' into add-multiexperiment

735762c

Rename

584091c

Merge branch 'add-multiexperiment' of github.com:smondal13/pyomo into…

3a52f2d

… add-multiexperiment

fix(doc): Test failure

3678ecb

- enhance documentation for init_solver usage in multi-experiment optimization

blnicho added the pyomo.DoE label Mar 18, 2026

		@@ -0,0 +1,362 @@
		# Review: `_DoEResultsJSONEncoder`, `_enum_label`, and `optimize_experiments()` API

Conversation

smondal13 commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixes # .

Summary/Motivation:

Changes proposed in this PR:

Note:

Legal Acknowledgement

Uh oh!

smondal13 Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

smondal13 Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smondal13 commented Mar 3, 2026

Uh oh!

smondal13 Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smondal13 Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smondal13 Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smondal13 commented Mar 6, 2026

Uh oh!

adowling2 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

smondal13 commented Mar 2, 2026 •

edited

Loading

smondal13 Mar 3, 2026 •

edited

Loading

smondal13 Mar 3, 2026 •

edited

Loading

smondal13 Mar 5, 2026 •

edited

Loading

smondal13 Mar 5, 2026 •

edited

Loading

smondal13 Mar 5, 2026 •

edited

Loading