Skip to content

Build pyOpenMS from source in Windows CI pipeline #313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
t0mdavid-m wants to merge 12 commits into
base: main
Choose a base branch
from
+98 −46
Open

Build pyOpenMS from source in Windows CI pipeline #313

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
7a81725
Build pyopenms from source in Windows executable workflow
claude Jan 22, 2026
dd8a1e5
Address CodeRabbit review comments
claude Jan 22, 2026
f663fb1
Fix workflow-tests.yml to install pyopenms separately
claude Jan 22, 2026
eef807b
Update numpy to 2.x for pyopenms 3.3.0+ compatibility
claude Jan 22, 2026
3002044
Update DataFrame column names for pyopenms 3.5.0 compatibility
claude Jan 22, 2026
7c13b76
Update DataFrame column names for pyopenms 3.5.0 compatibility
claude Jan 22, 2026
33c1dea
Update DataFrame column names for pyopenms 3.5.0 compatibility
claude Jan 22, 2026
e1707e8
Add explicit PYTHON_EXECUTABLE for pyopenms build
claude Jan 23, 2026
3dbcc3c
Add Python_FIND_STRATEGY for CMake FindPython module
claude Jan 23, 2026
2913429
Switch to Visual Studio generator for pyopenms build
claude Jan 23, 2026
2331566
Limit autowrap to <=0.24 to match Dockerfile
claude Jan 23, 2026
152725d
Add BUILD_TYPE to Test Windows step for VS generator
claude Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 56 additions & 4 deletions .github/workflows/build-windows-executable-app.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,16 @@ jobs:
# https://github.com/actions/runner-images/blob/main/images/win/scripts/Installers/Install-GitHub-CLI.ps1
echo "C:/Program Files (x86)/GitHub CLI" >> $GITHUB_PATH

- name: Set up Python for pyOpenMS build
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Install pyOpenMS build dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel "cython>=3.0.0" "autowrap<=0.24" "numpy>=1.24.0" pandas pytest

- name: Extract branch/PR infos
shell: bash
run: |
Expand Down Expand Up @@ -132,20 +142,28 @@ jobs:
env:
OPENMS_CONTRIB_LIBS: "${{ github.workspace }}/OpenMS/contrib"
CI_PROVIDER: "GitHub-Actions"
CMAKE_GENERATOR: "Ninja"
CMAKE_GENERATOR: "Visual Studio 17 2022"
CMAKE_GENERATOR_PLATFORM: "x64"
SOURCE_DIRECTORY: "${{ github.workspace }}/OpenMS"
BUILD_NAME: "${{ env.RUN_NAME }}-Win64-class-topp-${{ github.run_number }}"
ENABLE_STYLE_TESTING: "OFF"
ENABLE_TOPP_TESTING: "ON"
ENABLE_CLASS_TESTING: "ON"
WITH_GUI: "OFF"
WITH_PARQUET: "OFF"
WITH_PARQUET: "ON"
ADDRESS_SANITIZER: "Off"
BUILD_TYPE: "Release"
OPENMP: "Off"
USE_STATIC_BOOST: "On"
# BUILD_FLAGS: "-p:CL_MPCount=2" # For VS Generator and MSBuild
BUILD_FLAGS: "-j2" # Ninja will otherwise use all cores (doesn't go well in GHA)
# pyOpenMS build settings
PYOPENMS: "ON"
PY_NUM_THREADS: "2"
PY_MEMLEAK_DISABLE: "On"
PY_NO_OPTIMIZATION: "ON"
Python_ROOT_DIR: "${{ runner.tool_cache }}/Python/${{ env.PYTHON_VERSION }}/x64"
PYTHON_EXECUTABLE: "${{ runner.tool_cache }}/Python/${{ env.PYTHON_VERSION }}/x64/python.exe"
Python_FIND_STRATEGY: "LOCATION"
BUILD_FLAGS: "-p:CL_MPCount=2"
CMAKE_CCACHE_EXE: "ccache"
CCACHE_BASEDIR: ${{ github.workspace }}
CCACHE_DIR: ${{ github.workspace }}/.ccache
Expand All @@ -160,6 +178,22 @@ jobs:
SOURCE_DIRECTORY: "${{ github.workspace }}/OpenMS"
CI_PROVIDER: "GitHub-Actions"
BUILD_NAME: "${{ env.RUN_NAME }}-Win64-class-topp-${{ github.run_number }}"
BUILD_TYPE: "Release"

- name: Build pyOpenMS
shell: bash
run: |
cd $GITHUB_WORKSPACE/OpenMS/bld
cmake --build . --target pyopenms --config Release -- /p:CL_MPCount=2 /m:2

- name: Test pyOpenMS
shell: bash
run: |
cd $GITHUB_WORKSPACE/OpenMS/bld/pyOpenMS
pip install dist/*.whl
python -c "import pyopenms; print(f'pyOpenMS version: {pyopenms.__version__}')"
# Run basic tests (skip tutorial tests which may require additional data)
pytest tests/ -v --ignore=tests/test_tutorial.py -x -q 2>&1 | head -100
Comment on lines +189 to +196
Copy link

@coderabbitai coderabbitai bot Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Piping pytest output to head may mask test failures.

In bash, the exit code of a pipeline is the exit code of the last command by default. When pytest fails but head -100 succeeds, the step may pass despite test failures. Consider using set -o pipefail or capturing the exit code explicitly.

🔧 Suggested fix using pipefail 
     - name: Test pyOpenMS
       shell: bash
       run: |
+        set -o pipefail
         cd $GITHUB_WORKSPACE/OpenMS/bld/pyOpenMS
         pip install dist/*.whl
         python -c "import pyopenms; print(f'pyOpenMS version: {pyopenms.__version__}')"
         # Run basic tests (skip tutorial tests which may require additional data)
         pytest tests/ -v --ignore=tests/test_tutorial.py -x -q 2>&1 | head -100
🤖 Prompt for AI Agents 
In @.github/workflows/build-windows-executable-app.yaml around lines 186 - 193,
The CI step "Test pyOpenMS" currently pipes pytest output to head (the pytest
invocation: "pytest tests/ -v --ignore=tests/test_tutorial.py -x -q 2>&1 | head
-100"), which can hide failures because the pipeline return code is from the
last command; update the step to enable pipefail or capture pytest's exit code:
either add a shell option (e.g., run the step with "set -o pipefail" before
running pytest) or run pytest redirecting output to a temp file and then print
the head while explicitly exiting with pytest's exit code; apply this change in
the "Test pyOpenMS" run block so failures are propagated correctly.


- name: Package
shell: bash
Expand All @@ -178,6 +212,12 @@ jobs:
name: openms-package
path: ${{ github.workspace }}/OpenMS/bld/*.zip

- name: Upload pyOpenMS wheel as artifact
uses: actions/upload-artifact@v4
with:
name: pyopenms-wheel
path: ${{ github.workspace }}/OpenMS/bld/pyOpenMS/dist/*.whl

build-executable:
runs-on: windows-2022
needs: build-openms
Expand All @@ -199,6 +239,12 @@ jobs:
name: openms-package
path: openms-package

- name: Download pyOpenMS wheel
uses: actions/download-artifact@v4
with:
name: pyopenms-wheel
path: pyopenms-wheel

- name: Extract bin and share from package
run: |
cd openms-package
Expand Down Expand Up @@ -248,6 +294,12 @@ jobs:
- name: Install Required Packages
run: .\python-${{ env.PYTHON_VERSION }}\python -m pip install --force-reinstall -r requirements.txt --no-warn-script-location

- name: Install locally built pyOpenMS wheel
run: |
$wheel = Get-ChildItem -Path pyopenms-wheel -Filter "*.whl" | Select-Object -First 1
Write-Host "Installing pyOpenMS wheel: $($wheel.FullName)"
.\python-${{ env.PYTHON_VERSION }}\python -m pip install $wheel.FullName --no-warn-script-location

Comment on lines +297 to +302
Copy link

@coderabbitai coderabbitai bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fail fast when the wheel artifact is missing.

If no wheel exists, $wheel.FullName becomes empty and pip install fails with a confusing error. Consider an explicit guard.

🔧 Suggested fix 
     - name: Install locally built pyOpenMS wheel
       run: |
         $wheel = Get-ChildItem -Path pyopenms-wheel -Filter "*.whl" | Select-Object -First 1
+        if (-not $wheel) { throw "pyOpenMS wheel not found in pyopenms-wheel/" }
         Write-Host "Installing pyOpenMS wheel: $($wheel.FullName)"
         .\python-${{ env.PYTHON_VERSION }}\python -m pip install $wheel.FullName --no-warn-script-location
🤖 Prompt for AI Agents 
In @.github/workflows/build-windows-executable-app.yaml around lines 297 - 302,
The "Install locally built pyOpenMS wheel" step can attempt to install when no
wheel was found, producing a confusing pip error; add an explicit guard after
selecting $wheel (the result of Get-ChildItem | Select-Object -First 1) that
checks for null/empty and fails fast with a clear message (e.g.,
Write-Error/Write-Host then exit 1) so the job stops with a meaningful error
instead of passing an empty path to .\python-... -m pip install; ensure the
guard references the $wheel variable used later.

- name: Set to offline deployment
run: |
$content = Get-Content -Raw settings.json | ConvertFrom-Json
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/workflow-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pyopenms
Copy link

@coderabbitai coderabbitai bot Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Pin pyopenms version to ensure compatibility with column name changes.

The src/view.py changes use pyOpenMS 3.5.0 column naming (rt, mz_array, intensity_array). Installing without a version pin may install an older version with different column names (e.g., RT, mzarray, intarray), causing test failures.

🔧 Suggested fix 
-        pip install pyopenms
+        pip install "pyopenms>=3.5.0"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
pip install pyopenms
pip install "pyopenms>=3.5.0"
🤖 Prompt for AI Agents 
In @.github/workflows/workflow-tests.yml at line 22, The workflow installs
pyopenms without a version pin which can pull an incompatible release; update
the pip install step in the CI workflow (the line currently installing pyopenms)
to pin the version to the one used by your code (e.g., change to pip install
pyopenms==3.5.0) so src/view.py’s expected column names (rt, mz_array,
intensity_array) match the installed pyOpenMS API.

pip install pytest
- name: Running test cases
run: |
Expand Down
9 changes: 6 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,18 @@ To run the app locally:
```

2. **Install dependencies**

Make sure you can run ```pip``` commands.

Install all dependencies with:
```bash
pip install -r requirements.txt
pip install pyopenms
```

4. **Launch the app**
> ℹ️ **Note:** `pyopenms` is not included in `requirements.txt` because it is built from source for Docker and Windows executable distributions. For local development, install it separately via pip as shown above.

3. **Launch the app**
```bash
streamlit run app.py
```
Expand Down
8 changes: 2 additions & 6 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,16 +47,15 @@ kiwisolver==1.4.8
markupsafe==3.0.2
# via jinja2
matplotlib==3.10.1
# via pyopenms
# via src (pyproject.toml)
narwhals==2.5.0
# via altair
numpy==1.26.4
numpy>=2.0.0
# via
# contourpy
# matplotlib
# pandas
# pydeck
# pyopenms
# src (pyproject.toml)
# streamlit
packaging==25.0
Expand All @@ -67,7 +66,6 @@ packaging==25.0
# streamlit
pandas==2.2.3
# via
# pyopenms
# pyopenms-viz
# streamlit
pillow==11.1.0
Expand All @@ -85,8 +83,6 @@ pyarrow==19.0.1
# via streamlit
pydeck==0.9.1
# via streamlit
pyopenms==3.3.0
# via src (pyproject.toml)
pyopenms-viz==1.0.0
# via src (pyproject.toml)
pyparsing==3.2.3
Expand Down
66 changes: 33 additions & 33 deletions src/view.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ def get_df(file: Union[str, Path]) -> pd.DataFrame:

Returns:
pd.DataFrame: A pandas DataFrame with the following columns: "mslevel",
"precursormz", "mzarray", and "intarray". The "mzarray" and "intarray"
"precursormz", "mz_array", and "intensity_array". The "mz_array" and "intensity_array"
columns contain NumPy arrays with the m/z and intensity values for each
spectrum in the mzML file, respectively.
"""
Expand All @@ -37,10 +37,10 @@ def get_df(file: Union[str, Path]) -> pd.DataFrame:
df_spectra["precursor m/z"] = precs

# Drop spectra without peaks
df_spectra = df_spectra[df_spectra["mzarray"].apply(lambda x: len(x) > 0)]
df_spectra = df_spectra[df_spectra["mz_array"].apply(lambda x: len(x) > 0)]

df_spectra["max intensity m/z"] = df_spectra.apply(
lambda x: x["mzarray"][x["intarray"].argmax()], axis=1
lambda x: x["mz_array"][x["intensity_array"].argmax()], axis=1
)
if not df_spectra.empty:
st.session_state["view_spectra"] = df_spectra
Expand All @@ -56,11 +56,11 @@ def get_df(file: Union[str, Path]) -> pd.DataFrame:
if not exp_ms1.empty():
st.session_state["view_ms1"] = exp_ms1.get_df(long=True)
else:
st.session_state["view_ms1"] = pd.DataFrame(columns=['RT', 'mz', 'inty'])
st.session_state["view_ms1"] = pd.DataFrame(columns=['rt', 'mz', 'intensity'])
if not exp_ms2.empty():
st.session_state["view_ms2"] = exp_ms2.get_df(long=True)
else:
st.session_state["view_ms2"] = pd.DataFrame(columns=['RT', 'mz', 'inty'])
st.session_state["view_ms2"] = pd.DataFrame(columns=['rt', 'mz', 'intensity'])


def plot_bpc_tic() -> go.Figure:
Expand All @@ -72,31 +72,31 @@ def plot_bpc_tic() -> go.Figure:
fig = go.Figure()
max_int = 0
if st.session_state.view_tic:
df = st.session_state.view_ms1.groupby("RT").sum().reset_index()
df = st.session_state.view_ms1.groupby("rt").sum().reset_index()
df["type"] = "TIC"
if df["inty"].max() > max_int:
max_int = df["inty"].max()
if df["intensity"].max() > max_int:
max_int = df["intensity"].max()
fig = df.plot(
backend="ms_plotly",
kind="chromatogram",
x="RT",
y="inty",
x="rt",
y="intensity",
by="type",
color="#f24c5c",
show_plot=False,
grid=False,
aggregate_duplicates=True,
)
if st.session_state.view_bpc:
df = st.session_state.view_ms1.groupby("RT").max().reset_index()
df = st.session_state.view_ms1.groupby("rt").max().reset_index()
df["type"] = "BPC"
if df["inty"].max() > max_int:
max_int = df["inty"].max()
if df["intensity"].max() > max_int:
max_int = df["intensity"].max()
fig = df.plot(
backend="ms_plotly",
kind="chromatogram",
x="RT",
y="inty",
x="rt",
y="intensity",
by="type",
color="#2d3a9d",
show_plot=False,
Expand All @@ -118,13 +118,13 @@ def plot_bpc_tic() -> go.Figure:
].copy()
if not df_eic.empty:
df_eic.loc[:, "type"] = "XIC"
if df_eic["inty"].max() > max_int:
max_int = df_eic["inty"].max()
if df_eic["intensity"].max() > max_int:
max_int = df_eic["intensity"].max()
fig = df_eic.plot(
backend="ms_plotly",
kind="chromatogram",
x="RT",
y="inty",
x="rt",
y="intensity",
by="type",
color="#f6bf26",
show_plot=False,
Expand Down Expand Up @@ -174,17 +174,17 @@ def view_peak_map():
box = st.session_state.view_peak_map_selection.selection.box
if box:
df = st.session_state.view_ms1.copy()
df = df[df["RT"] > box[0]["x"][0]]
df = df[df["rt"] > box[0]["x"][0]]
df = df[df["mz"] > box[0]["y"][1]]
df = df[df["mz"] < box[0]["y"][0]]
df = df[df["RT"] < box[0]["x"][1]]
df = df[df["rt"] < box[0]["x"][1]]
if len(df) == 0:
return
peak_map = df.plot(
kind="peakmap",
x="RT",
x="rt",
y="mz",
z="inty",
z="intensity",
title=st.session_state.view_selected_file,
grid=False,
show_plot=False,
Expand All @@ -209,9 +209,9 @@ def view_peak_map():
kind="peakmap",
plot_3d=True,
backend="ms_plotly",
x="RT",
x="rt",
y="mz",
z="inty",
z="intensity",
zlabel="Intensity",
title="",
show_plot=False,
Expand All @@ -235,7 +235,7 @@ def view_spectrum():
df,
column_order=[
"spectrum ID",
"RT",
"rt",
"MS level",
"max intensity m/z",
"precursor m/z",
Expand All @@ -252,22 +252,22 @@ def view_spectrum():
box = st.session_state.view_spectrum_selection.selection.box
if box:
mz_min, mz_max = sorted(box[0]["x"])
mask = (df["mzarray"] > mz_min) & (df["mzarray"] < mz_max)
df["intarray"] = df["intarray"][mask]
df["mzarray"] = df["mzarray"][mask]
mask = (df["mz_array"] > mz_min) & (df["mz_array"] < mz_max)
df["intensity_array"] = df["intensity_array"][mask]
df["mz_array"] = df["mz_array"][mask]

if df["mzarray"].size > 0:
if df["mz_array"].size > 0:
title = f"{st.session_state.view_selected_file} spec={index+1} mslevel={df['MS level']}"
if df["precursor m/z"] > 0:
title += f" precursor m/z: {round(df['precursor m/z'], 4)}"

df_selected = pd.DataFrame(
{
"mz": df["mzarray"],
"intensity": df["intarray"],
"mz": df["mz_array"],
"intensity": df["intensity_array"],
}
)
df_selected["RT"] = df["RT"]
df_selected["rt"] = df["rt"]
df_selected["MS level"] = df["MS level"]
df_selected["precursor m/z"] = df["precursor m/z"]
df_selected["max intensity m/z"] = df["max intensity m/z"]
Expand Down
Loading