-
Notifications
You must be signed in to change notification settings - Fork 1
Update with template #18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
44415ba
c65d503
c81fee6
fb2fe67
128da6d
42fc187
6a2bc03
93170fa
b185cf0
8c03310
4069e11
eae014b
12f8e0d
0633790
aa585c2
7a2dc86
e7f6c08
fca45ee
6a2a7ae
d8d3d03
ebef5df
fa46191
dd32bd1
9ce9585
93fb2a4
29d94b4
e1cc9d4
1c183d0
bd9bf5e
859e481
8f25b59
f3c21a5
cf35e62
bbc43f4
25ed32c
072161a
d0a1fde
b970ce7
9933465
0e9ceca
69e655f
b28b1f1
e840358
636da9d
b69a2e5
594a83a
e5d8044
64dfed8
6f9c692
d78d8b8
8c6f869
2cb4813
971cfdd
6c61365
11ff5cd
5870c34
8abb90a
0bd2ccf
2f28ed9
43c300b
64f43e2
4ab1288
5bd5898
391ed16
3387b9c
0cce837
eeb8e3f
c627256
5901c90
03eff39
8e096ac
286fd0b
a732de0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| # Add Parameter Presets | ||
|
|
||
| Add or modify parameter presets for a TOPP workflow in `presets.json`. | ||
|
|
||
| ## Instructions | ||
|
|
||
| 1. **Ask the user** for: | ||
| - Which workflow the presets are for | ||
| - Preset names and descriptions | ||
| - Parameter values for each preset (TOPP tool parameters and/or custom widget parameters) | ||
|
|
||
| 2. **Determine the workflow key** from the display name: | ||
| - Convert to lowercase, replace spaces with hyphens | ||
| - Example: "TOPP Workflow" -> "topp-workflow" | ||
| - Example: "Metabolite Analysis" -> "metabolite-analysis" | ||
|
|
||
| 3. **Read or create `presets.json`** at the repository root. Add entries following this schema: | ||
|
|
||
| ```json | ||
| { | ||
| "workflow-name": { | ||
| "Preset Name": { | ||
| "_description": "Tooltip text shown on hover over the preset button", | ||
| "TOPPToolName": { | ||
| "algorithm:section:param_name": value | ||
| }, | ||
| "_general": { | ||
| "custom-widget-key": value | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| 4. **Verify the result** by checking that: | ||
| - Workflow name key matches the name passed to `WorkflowManager.__init__()` (lowercased, hyphenated) | ||
| - TOPP tool names match those used in `input_TOPP()` calls | ||
| - Parameter paths use colon-separated format matching the TOPP tool's .ini structure | ||
| - `_general` keys match widget keys from `input_widget()` calls | ||
| - JSON is valid | ||
|
|
||
| ## Schema Rules | ||
|
|
||
| - **Workflow key**: lowercase, hyphens — must match `WorkflowManager("Display Name", ...)` converted | ||
| - **`_description`**: optional tooltip text for the preset button | ||
| - **TOPP tool keys**: dictionary name must exactly match the TOPP tool name (e.g., `"FeatureFinderMetabo"`) | ||
| - **TOPP parameter paths**: colon-separated, e.g., `"algorithm:common:noise_threshold_int"` | ||
| - **`_general`**: overrides for custom `input_widget()` keys (non-TOPP parameters) | ||
| - **Keys starting with `_`** are metadata and not applied as tool parameters | ||
|
|
||
| ## How Presets Work at Runtime | ||
|
|
||
| 1. Preset buttons auto-appear in `parameter_section()` via `StreamlitUI.preset_buttons()` | ||
| 2. Only presets matching the current workflow name are displayed | ||
| 3. Clicking a preset updates `params.json` in the workspace and refreshes the UI | ||
| 4. If no `presets.json` exists or no presets match, no buttons are shown | ||
|
|
||
| ## Reference Files | ||
|
|
||
| - Existing presets: `presets.json` | ||
| - Preset loading: `src/workflow/ParameterManager.py` | ||
| - Preset buttons: `src/workflow/StreamlitUI.py` — `preset_buttons()` method | ||
| - Documentation: `docs/build_app.md` (Parameter Presets section) | ||
|
|
||
| ## Example | ||
|
|
||
| For a workflow initialized as `super().__init__("Feature Analysis", ...)`: | ||
|
|
||
| ```json | ||
| { | ||
| "feature-analysis": { | ||
| "High Sensitivity": { | ||
| "_description": "Optimized for detecting low-abundance features", | ||
| "FeatureFinderMetabo": { | ||
| "algorithm:common:noise_threshold_int": 500.0, | ||
| "algorithm:common:chrom_peak_snr": 2.0 | ||
| } | ||
| }, | ||
| "Fast Processing": { | ||
| "_description": "Quick analysis with relaxed thresholds", | ||
| "FeatureFinderMetabo": { | ||
| "algorithm:common:noise_threshold_int": 2000.0 | ||
| }, | ||
| "_general": { | ||
| "run-python-script": false | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Checklist | ||
|
|
||
| - [ ] Workflow key is lowercase with hyphens, matching the WorkflowManager name | ||
| - [ ] Each preset has a descriptive `_description` | ||
| - [ ] TOPP tool names match exactly | ||
| - [ ] Parameter paths use colon-separated format | ||
| - [ ] `presets.json` is valid JSON |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| # Add a Python Analysis Tool | ||
|
|
||
| Add a custom Python analysis script for MS data processing to `src/python-tools/` with auto-generated UI. | ||
|
|
||
| Python tools handle analysis steps that aren't covered by TOPP tools — e.g., exporting consensus features to DataFrames, computing differential expression statistics, custom metabolite annotation, or post-processing identification results. | ||
|
|
||
| ## Instructions | ||
|
|
||
| 1. **Ask the user** for: | ||
| - Tool name (lowercase, hyphens for spaces) | ||
| - What the tool does | ||
| - Input/output file types | ||
| - Parameters: name, type, default value, constraints | ||
|
|
||
| 2. **Create the tool script** at `src/python-tools/<name>.py`: | ||
|
|
||
| ```python | ||
| import json | ||
| import sys | ||
|
|
||
| DEFAULTS = [ | ||
| # Hidden I/O parameters (always include these) | ||
| {"key": "in", "value": [], "help": "Input files.", "hide": True}, | ||
| {"key": "out", "value": [], "help": "Output files.", "hide": True}, | ||
| # Visible parameters — examples of each type: | ||
| { | ||
| "key": "threshold", | ||
| "value": 1000.0, | ||
| "name": "Intensity Threshold", | ||
| "help": "Minimum intensity for filtering.", | ||
| "min": 0.0, | ||
| "max": 100000.0, | ||
| "step_size": 100.0, | ||
| }, | ||
| { | ||
| "key": "method", | ||
| "value": "default", | ||
| "name": "Processing Method", | ||
| "options": ["default", "advanced", "custom"], | ||
| "help": "Which algorithm to use.", | ||
| }, | ||
| { | ||
| "key": "num-features", | ||
| "value": 10, | ||
| "name": "Number of Features", | ||
| "widget_type": "slider", | ||
| "min": 1, | ||
| "max": 100, | ||
| "step_size": 1, | ||
| "help": "How many features to report.", | ||
| }, | ||
| { | ||
| "key": "advanced-setting", | ||
| "value": 5, | ||
| "name": "Advanced Setting", | ||
| "help": "Only shown in advanced mode.", | ||
| "advanced": True, | ||
| }, | ||
| { | ||
| "key": "enabled", | ||
| "value": True, | ||
| "name": "Enable Processing", | ||
| }, | ||
| ] | ||
|
|
||
|
|
||
| def get_params(): | ||
| if len(sys.argv) > 1: | ||
| with open(sys.argv[1], "r") as f: | ||
| return json.load(f) | ||
| else: | ||
| return {} | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| params = get_params() | ||
| # Tool logic here — read inputs, process, write outputs | ||
| ``` | ||
|
|
||
| 3. **Wire into a workflow** by adding to the workflow's `configure()` and `execution()` methods: | ||
|
|
||
| ```python | ||
| # In configure(): | ||
| self.ui.input_python("tool_name") | ||
|
|
||
| # In execution(): | ||
| self.executor.run_python("tool_name", {"in": input_files}) | ||
| ``` | ||
|
|
||
| ## DEFAULTS Metadata Keys | ||
|
|
||
| | Key | Required | Type | Description | | ||
| |-----|----------|------|-------------| | ||
| | `key` | Yes | str | Unique identifier for the parameter | | ||
| | `value` | Yes | any | Default value (type determines widget: bool=checkbox, str with options=selectbox, number=number_input) | | ||
| | `name` | No | str | Display name in the UI | | ||
| | `help` | No | str | Tooltip text | | ||
| | `hide` | No | bool | If `True`, parameter is not shown in UI (use for in/out files) | | ||
| | `options` | No | list | Valid choices — renders as selectbox | | ||
| | `min` | No | number | Minimum value for numeric inputs | | ||
| | `max` | No | number | Maximum value for numeric inputs | | ||
| | `step_size` | No | number | Step size for numeric inputs | | ||
| | `widget_type` | No | str | Override widget type: `"slider"`, `"textarea"`, `"number"`, `"text"`, etc. | | ||
| | `advanced` | No | bool | If `True`, only shown when user expands advanced parameters | | ||
|
|
||
| ## Reference Files | ||
|
|
||
| - Example tool: `src/python-tools/example.py` | ||
| - Another example: `src/python-tools/export_consensus_feature_df.py` | ||
| - UI generation: `src/workflow/StreamlitUI.py` — `input_python()` method | ||
| - Execution: `src/workflow/CommandExecutor.py` — `run_python()` method | ||
|
|
||
| ## Checklist | ||
|
|
||
| - [ ] Script created in `src/python-tools/` with DEFAULTS list | ||
| - [ ] `get_params()` function and `__main__` block included | ||
| - [ ] `in` and `out` keys in DEFAULTS with `"hide": True` | ||
| - [ ] Wired into workflow via `self.ui.input_python()` and `self.executor.run_python()` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| # Add MS Data Visualization | ||
|
|
||
| Add mass spectrometry data visualizations using pyopenms-viz or OpenMS-Insight components. | ||
|
|
||
| MS data has specialized visualization needs: mass spectra (m/z vs intensity stick plots), chromatograms (RT vs intensity), 2D peak maps (RT vs m/z heatmaps), isotope patterns, fragment ion annotations, and statistical plots like volcano plots for differential expression. | ||
|
|
||
| ## Instructions | ||
|
|
||
| 1. **Ask the user** for: | ||
| - What data to visualize (spectra, chromatograms, peak maps, tables, heatmaps) | ||
| - Data size expectations (small/medium vs. large datasets with millions of points) | ||
| - Whether interactive cross-component linking is needed | ||
|
|
||
| 2. **Choose the right library**: | ||
|
|
||
| | Use Case | Library | Why | | ||
| |----------|---------|-----| | ||
| | Quick spectrum/chromatogram/peak map plots | **pyopenms-viz** | Simple one-liner API, publication quality | | ||
| | Large datasets (millions of points) | **OpenMS-Insight** | Server-side pagination, intelligent downsampling | | ||
| | Interactive tables with pagination | **OpenMS-Insight** `Table` | Tabulator.js, CSV export, server-side filtering | | ||
| | Cross-linked views (click table row → highlight in plot) | **OpenMS-Insight** | Shared `link_id` across components | | ||
| | Standard Plotly plots (bar, scatter, etc.) | **plotly.express** directly | Already available, use with `show_fig()` | | ||
|
|
||
| 3. **Implement the visualization** in a page or workflow results section. | ||
|
|
||
| ## pyopenms-viz Patterns | ||
|
|
||
| pyopenms-viz extends pandas DataFrames with MS-specific plot accessors. Always use the `plotly` backend for Streamlit. | ||
|
|
||
| ```python | ||
| import pyopenms_viz | ||
| from src.common.common import show_fig | ||
|
|
||
| # Mass spectrum (stick plot from mz/intensity columns) | ||
| fig = df.plot.ms_spectrum(backend="plotly", title="MS1 Spectrum") | ||
| show_fig(fig, "spectrum-plot") | ||
|
|
||
| # 2D peak map (RT vs m/z, colored by intensity) | ||
| fig = df.plot.peak_map(backend="plotly") | ||
| show_fig(fig, "peak-map") | ||
|
|
||
| # Chromatogram (RT vs intensity) | ||
| fig = df.plot.chromatogram(backend="plotly") | ||
| show_fig(fig, "chromatogram") | ||
|
|
||
| # Mobilogram (ion mobility vs intensity) | ||
| fig = df.plot.mobilogram(backend="plotly") | ||
| show_fig(fig, "mobilogram") | ||
| ``` | ||
|
|
||
| **Key points:** | ||
| - DataFrame must have appropriate columns (e.g., `mz`, `intensity` for spectra) | ||
| - Always use `backend="plotly"` in Streamlit context | ||
| - Use `show_fig()` from `src/common/common.py` for consistent display with download buttons | ||
| - Import `pyopenms_viz` to register the plot accessors (even if not used directly) | ||
|
|
||
| ## OpenMS-Insight Patterns | ||
|
|
||
| OpenMS-Insight provides Vue.js-backed interactive components optimized for large MS datasets. | ||
|
|
||
| ```python | ||
| from openms_insight import Table, LinePlot, Heatmap, VolcanoPlot, SequenceView | ||
|
|
||
| # Interactive table with server-side pagination | ||
| Table(df, key="results-table", page_size=50) | ||
|
|
||
| # Stick-style mass spectrum | ||
| LinePlot(df, x="mz", y="intensity", key="spectrum-plot") | ||
|
|
||
| # 2D heatmap for large datasets (auto-downsampling) | ||
| Heatmap(df, x="rt", y="mz", z="intensity", key="peak-heatmap") | ||
|
|
||
| # Volcano plot for differential expression | ||
| VolcanoPlot(df, x="log2fc", y="neg_log10_pval", key="volcano") | ||
|
|
||
| # Peptide sequence with fragment ion annotations | ||
| SequenceView(sequence="PEPTIDER", ions=ion_df, key="sequence") | ||
| ``` | ||
|
|
||
| ### Cross-Component Linking | ||
|
|
||
| Link components so selections in one update another: | ||
|
|
||
| ```python | ||
| # Selecting a row in the table highlights the corresponding point in the plot | ||
| Table(df, key="linked-table", link_id="feature_id") | ||
| LinePlot(df, x="mz", y="intensity", key="linked-plot", link_id="feature_id") | ||
| Heatmap(df, x="rt", y="mz", z="intensity", key="linked-heatmap", link_id="feature_id") | ||
| ``` | ||
|
|
||
| All components sharing the same `link_id` column are automatically synchronized. | ||
|
Comment on lines
+84
to
+91
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct the cross-component linking pattern to use The documented pattern shows using a ✅ Correct linking patternfrom openms_insight import Table, LinePlot, Heatmap, StateManager
# Initialize state manager ONCE for all linked components
state_manager = StateManager()
# Initialize components from cache
table = Table(cache_id="table_id", cache_path=str(cache_dir))
line_plot = LinePlot(cache_id="lineplot_id", cache_path=str(cache_dir))
heatmap = Heatmap(cache_id="heatmap_id", cache_path=str(cache_dir))
# Render components with the SAME state_manager instance
table(state_manager=state_manager, height=533)
line_plot(state_manager=state_manager, height=450)
heatmap(state_manager=state_manager, height=350)Selections in one component automatically update all components sharing the same 🤖 Prompt for AI Agents |
||
|
|
||
| ## Integration in Workflow Results | ||
|
|
||
| Add visualization to a workflow's `results()` method: | ||
|
|
||
| ```python | ||
| @st.fragment | ||
| def results(self) -> None: | ||
| result_file = Path(self.workflow_dir, "results", "step-name", "output.tsv") | ||
| if result_file.exists(): | ||
| df = pd.read_csv(result_file, sep="\t") | ||
|
|
||
| # pyopenms-viz for spectrum plots | ||
| import pyopenms_viz | ||
| fig = df.plot.ms_spectrum(backend="plotly") | ||
| show_fig(fig, "results-spectrum") | ||
|
|
||
| # Or OpenMS-Insight for interactive exploration | ||
| from openms_insight import Table | ||
| Table(df, key="results-table") | ||
| else: | ||
| st.warning("No results found. Please run the workflow first.") | ||
| ``` | ||
|
Comment on lines
+98
to
+114
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Align the workflow integration example with the cache-based pattern. The OpenMS-Insight example in the 🔄 Suggested alignment with actual pattern`@st.fragment`
def results(self) -> None:
cache_dir = Path(self.workflow_dir, "results", "insight_cache")
# pyopenms-viz for spectrum plots
result_file = Path(self.workflow_dir, "results", "step-name", "output.tsv")
if result_file.exists():
df = pd.read_csv(result_file, sep="\t")
import pyopenms_viz
fig = df.plot.ms_spectrum(backend="plotly")
show_fig(fig, "results-spectrum")
# OpenMS-Insight for interactive exploration (cache-based)
cache_id = "results_table"
if (cache_dir / f"table_{cache_id}").is_dir():
from openms_insight import Table, StateManager
state_manager = StateManager()
table = Table(cache_id=f"table_{cache_id}", cache_path=str(cache_dir))
table(state_manager=state_manager, height=533)
else:
st.warning("No cached results found. Please run the workflow first.")Alternatively, if direct data passing is a valid alternative pattern, clarify when to use each approach. 🤖 Prompt for AI Agents |
||
|
|
||
| ## Reference Files | ||
|
|
||
| - Display utilities: `src/common/common.py` — `show_fig()`, `show_table()` | ||
| - Dependencies: `requirements.txt` — `pyopenms-viz`, `openms-insight` | ||
| - Example workflow results: `src/Workflow.py` — `results()` method | ||
|
|
||
| ## Library Repositories | ||
|
|
||
| - **pyopenms-viz**: OpenMS/pyopenms-viz — pandas DataFrame `.plot()` extension with matplotlib/bokeh/plotly backends | ||
| - **OpenMS-Insight**: t0mdavid-m/openms-insight — Vue.js interactive Streamlit components with caching and downsampling | ||
|
|
||
| ## Checklist | ||
|
|
||
| - [ ] Correct library chosen for the use case | ||
| - [ ] Dependencies present in `requirements.txt` | ||
| - [ ] `show_fig()` used for pyopenms-viz plots (consistent download/export behavior) | ||
| - [ ] `backend="plotly"` specified for all pyopenms-viz plots | ||
| - [ ] Unique `key=` values for all OpenMS-Insight components | ||
| - [ ] Cross-component linking set up if multiple views of same data | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct the OpenMS-Insight API pattern to match the actual implementation.
The documented pattern shows direct data passing (
Table(df, key="results-table")), but the actual implementation in the codebase uses a cache-based approach. Components are initialized withcache_idandcache_path, then called withstate_managerand other parameters.✅ Correct pattern from content/results_database_search.py
Key differences:
cache_idandcache_path(not data)state_manageris passed when rendering, notlink_idkeyis passed during rendering, not initialization🤖 Prompt for AI Agents