Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
7c29cd8
Updates to build: allow argparse (and integration with CLion)
CodeByDrescher Jun 25, 2025
f3c3107
[WIP] Cleaned up most warnings
CodeByDrescher Jun 25, 2025
b1be7be
Converted to Argparse entry
CodeByDrescher Jun 25, 2025
9418160
Moved input-reading outside of solver
CodeByDrescher Jun 30, 2025
096ba48
more sprintf removal
CodeByDrescher Jun 30, 2025
b90b008
Migrated Tests to C++, fixed message param parsing
CodeByDrescher Nov 13, 2025
8021892
Made messaging vs non-messaging more flexible
CodeByDrescher Nov 19, 2025
ed75ea2
Create separate functionality classes
CodeByDrescher Dec 9, 2025
100e302
Wire in new classes into workflow; add tests
CodeByDrescher Dec 9, 2025
a8cc4ed
downgraded j-thread for better compatability
CodeByDrescher Dec 9, 2025
c1c0294
Fixed remaining memory leaks! Now leak free!
CodeByDrescher Dec 10, 2025
180ddd8
Added catch to messaging loop, should give better message and stay alive
CodeByDrescher Dec 16, 2025
d2fd729
TSAN caught data race; added protective locks
CodeByDrescher Dec 16, 2025
8f69da7
simplified event-queue, and fixed mutex-access-before-constructor-fin…
CodeByDrescher Dec 17, 2025
f1f1cb7
rearranged some test code to use same body
CodeByDrescher Dec 17, 2025
1a306e5
Fixed undefined behavior bug
CodeByDrescher Dec 17, 2025
dd2eddf
trying libcurl downgrade
CodeByDrescher Nov 13, 2025
3a0cdb3
changing intel-mac conan installation to python
CodeByDrescher Nov 13, 2025
2ded3e3
intel mac 13 too old; upgrading to MacOS15!
CodeByDrescher Nov 13, 2025
c2ffa5d
Updating build configurations
CodeByDrescher Dec 9, 2025
0802000
Added build argument to select whether to test with localhost or not
CodeByDrescher Dec 9, 2025
91ee797
Added logging library
CodeByDrescher Dec 10, 2025
9581e7a
fixed upload paths
CodeByDrescher Dec 10, 2025
fd95e24
updating universal libraries code
CodeByDrescher Dec 11, 2025
313158d
Upgrading cmake to allow for all sanatizers + removed extra folder up…
CodeByDrescher Dec 17, 2025
baf7586
removed int-overflow in test
CodeByDrescher Dec 18, 2025
deecdbc
Add pybind11
CodeByDrescher Apr 22, 2026
cf6c5f9
Add python bindings and CD action
CodeByDrescher Apr 22, 2026
f81042f
Fix issues with CD
CodeByDrescher Apr 22, 2026
6ab54bb
Add Python smoke test and refresh CLAUDE.md
jcschaff May 6, 2026
56ca2a1
made testing clearer & clarified cmake limitations
CodeByDrescher May 7, 2026
e627fa4
removing commenting and reordering
CodeByDrescher May 8, 2026
9ba92c0
remove unused, old method
CodeByDrescher May 8, 2026
52b1485
Fix bad re-ordering
CodeByDrescher May 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 3 additions & 2 deletions .github/scripts/install_name_tool_macos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,12 @@ shopt -s -o nounset
for exe in `ls *_x64`
do
echo "fixing paths in ${exe}"
for libpath in `otool -L ${exe} | grep "/" | grep -v "System" | awk '{print $1}'`
for libpath in `otool -L ${exe} | grep "/" | grep -v "/usr/lib" | grep -v "System" | awk '{print $1}'`
do
libfilename=${libpath##*/}
echo install_name_tool -change $libpath @executable_path/$libfilename $exe
install_name_tool -change $libpath @executable_path/$libfilename $exe
cp libpath "$(pwd)"
done
done

Expand All @@ -29,7 +30,7 @@ do
echo install_name_tool -id "@loader_path/$libfilename" $libfilename
install_name_tool -id "@loader_path/$libfilename" $libfilename

for dependentlibpath in `otool -L ${libfilename} | grep "/" | grep -v "System" | awk '{print $1}'`
for dependentlibpath in `otool -L ${libfilename} | grep "/" | grep -v "/usr/lib" | grep -v "System" | awk '{print $1}'`
do
dependentlibfilename=${dependentlibpath##*/}
#
Expand Down
584 changes: 453 additions & 131 deletions .github/workflows/cd.yml

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions .github/workflows/docker-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ env:
jobs:
build-and-push-image:
runs-on: ubuntu-latest
services:
activemq:
image: rmohr/activemq:latest # Or a specific version like rmohr/activemq:5.15.9
ports:
- 30163:30163 # JMS port
- 8161:8161 # Web console port
permissions:
contents: read
packages: write
Expand Down
36 changes: 20 additions & 16 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,30 +1,34 @@
/build-linux64-server/
/build-dockcross-win64/

/.project
/.cproject
/build-win64/
/.DS_Store
/build-linux64/
/build-macos/
/packages/
/build_archive/
/build-linux64-ubuntu/

/dist
/conan/
/singularity/
/cmake-build*
/build*
/debug-build*
/.idea/
/nfsim/
/all_solvers/*

.DS_Store

NFsim_v1.11/tests/smoke/SimID_273069657_0_.gdat
NFsim_v1.11/tests/smoke/SimID_273069657_0_.species

IDAWin/tests/smoke/SimID_1489333437_0_.ida
*.ida

CMakeUserPresets.json

/build-linux64-server/
/build-dockcross-win64/
/build-win64/
/build-linux64/
/build-macos/
/build_archive/
/build-linux64-ubuntu/
/build*
/cmake-build*
/debug-build*
conan-build
conan-build/*
conan_provider.cmake

tests/unit/__pycache__/

wheelhouse/
110 changes: 110 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this repo is

`vcell-ode` is the ODE/DAE numerical solver used by the Virtual Cell framework (https://github.com/virtualcell/vcell). It produces two artifacts from one source tree:

- **A standalone CLI** (`SundialsSolverStandalone_x64`) that VCell launches as a subprocess against a `.cvodeInput` file.
- **A Python wheel** (`pyvcell_odesolver`) exposing the same `solve()` entry through a pybind11 module (`pyvcell_odesolver._core`).

The numerical core is CVODE/IDA from a vendored copy of `sundials/`, wrapped by `IDAWin/`, with user rate laws evaluated via `ExpressionParser/`. Optional progress messaging (libcurl → ActiveMQ over the JMS REST bridge) is provided by `VCellMessaging/`.

## Build

CMake-driven. Conan 2.x manages dependencies (`argparse`, `spdlog`, optionally `libcurl`); pybind11 is vendored at `extern/pybind11/`. Canonical native build (matches `.github/workflows/cd.yml`):

```bash
conan install . --output-folder build --build=missing
cd build && source conanbuild.sh
cmake -B . -S .. -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE=Release
cmake --build . --config Release
ctest -VV
```

Wheel build (separate scikit-build-core invocation; forces messaging OFF):

```bash
python3 -m build --wheel -o ./wheelhouse \
-C cmake.args="-DCMAKE_TOOLCHAIN_FILE=$PWD/build/conan_toolchain.cmake" \
-C cmake.define.OPTION_TARGET_PYTHON_BINDING=ON
```

The complete option set in the root `CMakeLists.txt`:

- `OPTION_TARGET_PYTHON_BINDING` (default OFF) — switches the project to scikit-build-core mode and builds the `_core` pybind11 extension. **Forces `OPTION_TARGET_MESSAGING=OFF`** and skips the `bin/` output-dir setup, so the standalone exe is not produced in this mode.
- `OPTION_TARGET_MESSAGING` (default OFF) — adds `-DUSE_MESSAGING`, links libcurl, and changes the install layout (`OPTION_EXE_DIRECTORY` → `../bin`). Without it, `VCellMessaging` still builds, but the curl path is replaced by `NullCurlProxy` (the `AbstractCurlProxy` polymorphism — not `#ifdef` forests).
- `OPTION_TEST_WITH_LOCALHOST` — adds `-DTEST_WITH_LOCALHOST`; flips `tests/unit/smoke_test.cpp` to the localhost-keyed expected-output baseline.
- `OPTION_STATICALLY_LINK`, `OPTION_TARGET_DOCS`, `OPTION_EXTRA_CONFIG_INFO` — niche flags; rarely flipped.

Conan profiles for CI live under `conan-profiles/CI-CD/`, one per `(platform, arch)`. They pin `compiler.cppstd=20`, `compiler.libcxx=libc++`, and add `mold` for Linux exe linking. For local dev, `conan profile detect --force` works on macOS/Linux as long as you bump `cppstd` to `20` to match the project.

Platform notes (full recipes in `cd.yml`):
- **Linux**: clang + `mold` (linker), `libc++-dev`, `libc++abi-dev`, conan installs everything else.
- **macOS** (arm64 + x86_64): Homebrew `conan` + `spdlog`. arm64 and x86_64 wheels build separately; a `MacOS-Universal` job `lipo`s the resulting `_x64`/`.dylib` files.
- **Windows**: LLVM toolchain via `llvm/actions/setup-windows`, conan + chocolatey. `OPTION_TARGET_MESSAGING` is OFF on Windows (libcurl path is not built).
- **Docker** (`Dockerfile`): Debian trixie-slim + clang-19 + libc++-19 + mold + ninja + cmake + spdlog from apt; builds with `OPTION_TARGET_MESSAGING=ON` and `OPTION_TEST_WITH_LOCALHOST=ON`. This is the image published to ghcr.io.

## Tests

`ctest -VV` from `build/` runs four C++ tests built into `build/bin/unit_tests`, plus optionally a Python test:

- `SmokeTest.UserProvidesFilesWithoutJMS` / `SmokeTest.UserProvidesFilesWithJMS` (`tests/unit/smoke_test.cpp`) — runs CVODE against `tests/unit/resources/SimID_*_.cvodeInput` and diffs the output against `*.ida.expected` with relative tolerance `1e-7`. The "WithJMS" variant uses `taskID=2025` and selects the `886118677` (localhost) or `256118677` (vcell.cam.uchc.edu) baseline based on `TEST_WITH_LOCALHOST`. Both variants pass with messaging off — the solver no-ops the curl path under `NullCurlProxy`.
- `HelloTest.BasicAssertions` (`tests/unit/hello_test.cpp`) — gtest sanity.
- `MessageProcessingTest.MessagesAreProcessed` (`tests/unit/message_processing_test.cpp`) — exercises `MessageEventManager` directly (worker thread + lock-ordered queue), independent of `OPTION_TARGET_MESSAGING`.
- `python_smoke_test` (`tests/unit/smoke_test.py`) — pytest, exercises `pyvcell_odesolver.solve()` against the same `1489333437` resource the C++ smoke uses. Registration is **skipped at configure time** unless `${PYTHON_TEST_INTERPRETER}` can `import pyvcell_odesolver`. To enable it locally: build a wheel, install it into a venv, then re-run `cmake -DPYTHON_TEST_INTERPRETER=<venv>/bin/python .`. CI's native ctest job uses the system Python so the entry skips silently; the wheel is exercised separately by `cibuildwheel` (which runs `pytest {project}/tests` per `pyproject.toml`, picking up both `tests/test_basic.py` and `tests/unit/smoke_test.py` by default discovery).

Run a single test: `ctest -VV -R <regex>`. Smoke-test inputs live next to the script and are baselined — if you intentionally change numerical output, regenerate the `.expected` files.

`ExpressionParserTest/ExpressionParserTest.cpp` exists but is not currently wired into ctest.

## Produced binaries

In native builds, `build/bin/` contains:
- `SundialsSolverStandalone_x64` — the CLI VCell invokes; `argparse`-driven, takes `<input> <output> [-tid <jobId>]`.
- `unit_tests` — gtest binary with the C++ tests above.

In Python-binding builds, the wheel includes `pyvcell_odesolver/_core.<soabi>.so` (or `.pyd` on Windows). The Python API exposes:
- `pyvcell_odesolver.version() -> str`
- `pyvcell_odesolver.solve(cvode_input_file_path, output_file_path, tid=-1) -> int` — `tid=-1` skips messaging; positive values are JMS job IDs.

## Architecture

Top-level subdirectories and how they fit:

- **`ExpressionParser/`** — math expression parser (AST `Node`s, `Expression`, `SymbolTable`, `StackMachine`). Used by `IDAWin` to evaluate user-supplied rate laws, initial conditions, and event/discontinuity expressions.
- **`sundials/`** (vendored, modified) — CVODE + IDA + nvec_ser. A thin top-level `add_subdirectory` aggregates `sundials_cvode`/`sundials_ida`/`sundials_nvecserial`/`sundials_lib` into the `sundials` interface target.
- **`IDAWin/`** — the solver layer. Decomposed (since the stabilization branch) into:
- `VCellSolver.h` — abstract base (`configureFromInput`, `solve`).
- `VCellSolverInput.{h,cpp}` — `VCellSolverInputBreakdown` struct-of-structs holding `ModelSettings`, `TimeCourseSettings`, `SteadyStateSettings`, `DiscontinuitiesSettings`, `EventSettings`.
- `VCellSolverFactory.{h,cpp}` — parses the `.cvodeInput` file into the breakdown and produces the right `VCellSolver` subclass.
- `VCellSundialsSolver.{h,cpp}` — common sundials machinery (events, discontinuities, output, stop-checking).
- `VCellCVodeSolver` / `VCellIDASolver` — concrete subclasses for ODE and DAE.
- `SundialsSolverInterface.{h,cpp}` — the `solve(input, output, tid)` entry shared by the CLI and the Python module.
- `SundialsSolverStandalone.cpp` — `argparse`-based `main` that calls `solve()`.
- **`VCellMessaging/`** — progress messaging. Decomposed into:
- `SimulationMessaging` — the singleton facade.
- `MessageEventManager` — owns the worker `std::thread` and the event queue; lock ordering documented in the header (`stopRequested` mutex *before* the queue mutex).
- `CurlProxyClasses` — `AbstractCurlProxy` / `NullCurlProxy` (always built) / `CurlProxy` (only under `USE_MESSAGING`); polymorphism replaces the old `#ifdef` forests.
- `JobEventStatus` — `JobEvent::Status` namespaced enum + `toString`.
- `WorkerEvent` — the queue payload.
- **`src/main.cpp`** — pybind11 module (`PYBIND11_MODULE(_core, m)`) exposing `version` and `solve` to Python.
- **`tests/`** — gtest C++ tests, pytest Python tests, and resource files. `RESOURCE_DIR` is passed to the C++ tests via `target_compile_definitions`; the Python test resolves resources via `Path(__file__).parent / "resources"`.
- **`extern/pybind11/`** — vendored pybind11. Used as the fallback when `find_package(pybind11)` doesn't find a system install.
- **`conan-profiles/CI-CD/`** — per-platform Conan profiles; CI copies the matching one to `~/.conan2/profiles/default`.
- **`cmake/modules/`** — `GetGitRevisionDescription.cmake` (used to stamp `g_GIT_DESCRIBE` into `VCellMessaging/src/GitDescribe.cpp`, which the version string surfaces).

Public headers for shared libraries live in their own dir and are pulled in via `target_include_directories(... PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})` (or `INTERFACE include/` for `VCellMessaging`/`sundials`) — there are no installed include dirs at configure time.

## Repo conventions worth knowing

- Output dir is `build/bin/` for native builds (set unconditionally when `OPTION_TARGET_PYTHON_BINDING=OFF`); the wheel build's output dir is whatever scikit-build-core picks (a tmpdir under `/var/folders/...` on macOS).
- `cmake-build-debug/` is the CLion out-of-source build dir; `build/` is the canonical CI/Docker dir. Both are gitignored.
- `extern/pybind11/` and `sundials/` are vendored; prefer surgical edits over upstream re-syncs.
- Shared-library path fixup for releases is done post-build by `.github/scripts/install_name_tool_macos.sh` (macOS) and the `ldd | cp` loops in `cd.yml` (Linux/Windows). Don't introduce absolute `rpath`s into CMake.
- `OPTION_TARGET_PYTHON_BINDING=ON` and `OPTION_TARGET_MESSAGING=ON` are mutually exclusive at the CMake level (the former forces the latter off). The Python wheel never carries a curl dependency.
- `pyproject.toml` is the source of truth for the Python package: `requires-python = ">=3.10"`, `testpaths = ["tests"]`, `filterwarnings = ["error"]` (any warning under pytest is a test failure), and `cibuildwheel` runs `pytest {project}/tests` against the installed wheel.
- Setting `-DOPTION_EXTRA_CONFIG_INFO=ON` dumps every CMake variable at the end of configure — handy for debugging Conan-generated targets.
Loading
Loading