Skip to content

init_solver_benchmark.md

Latest commit

 

History

History
67 lines (55 loc) · 3.48 KB

init_solver_benchmark.md

File metadata and controls

67 lines (55 loc) · 3.48 KB

Cross-solver ftINIT benchmark — Human-GEM / HCT116

Same ftinit() call (no-task scaled prep; mip_gap=0.001, time_limit=900s) run with each installed MILP-capable optlang interface. Generated by scripts/analyze_init_solvers.py; companion to the CI-scale tests/test_init_solvers.py.

Per-solver result

solver time (s) status n_rxns
gurobi 518 ✅ ok 7752
hybrid (HiGHS) 55 ❌ FAIL: ValueError: LP Method primal is not valid (choose one of: auto, simplex, interior point) 0
glpk 3672 ❌ FAIL: did not converge in 1 h+ (configuration.timeout not honored by GLPK MIP) 0

Wall clocks on Gurobi 13.0.1, optlang 1.x, cobra; one Human-GEM HCT116 cell line.

Findings

  • Gurobi is the only MILP backend that actually completes ftINIT on Human-GEM here: ~9 min for 7752 reactions (matches the validation result). All our tractability tuning (big-M=100, rescaleModelForINIT, mip_gap, time_limit) was done on Gurobi and it pays off.
  • HiGHS (hybrid_interface) does not work with cobra at all in this stack — not raven-python's bug. Cobra sets model.solver = "hybrid" which calls optlang.interface.Model.clone(), which re-applies a stored lp_method="primal" parameter that the hybrid_interface.Configuration rejects (it accepts only auto/simplex/interior point). This breaks model.copy() and any flow that swaps the solver — i.e. the whole pipeline. The same failure mode shows up at toy scale in tests/test_init_solvers.py (5/5 fail), so CI catches it now. Upstream optlang/cobra patch needed; nothing to fix in raven-python.
  • GLPK loads the model but its MIP solver does not honor configuration.timeout for this problem — we set the 900 s wall limit, GLPK still ran 1 h+ at 100 % CPU without producing a solution and had to be killed. GLPK has no licensing burden but is not a viable MILP backend at genome scale for ftINIT in practice.

Practical implications

  • Production / genome-scale ftINIT requires Gurobi today. We should be explicit about this in the package docs (license-encumbered dependency) until either the optlang hybrid_interface clone bug is fixed or GLPK gains usable MIP time-limit support.
  • Toy / unit-test correctness is portable. tests/test_init_solvers.py shows Gurobi and GLPK give identical verdicts on the toy ftINIT/tINIT networks; the formulation itself is solver-independent. Local development and CI work without a Gurobi license; only the genome-scale runs need it.
  • Future portability work is two concrete upstream fixes:
    1. optlang hybrid_interface.Configuration should accept (or remap) the lp_method parameter values that the generic clone path emits, or the clone path should drop unknown LP-method values gracefully.
    2. GLPK's MIP solve should honor configuration.timeout. If upstream won't, raven-python could implement a watchdog (separate thread sending SIGINT after the wall limit) specifically when the solver is GLPK.

Reproducing

# CI parameterised tests (seconds, runs always):
python -m pytest tests/test_init_solvers.py -v

# Genome-scale benchmark (minutes-to-hours, manual):
python scripts/analyze_init_solvers.py --cell HCT116 \
    --doc docs/init_solver_benchmark.md

Both reuse the cached Human-GEM no-task prep from the validation run (humangem_validation.md) and are resumable per solver.