Implement unified pipeline for orchestrating bulk and single-cell RNA-seq analyses

[Copilot]

Created a combined pipeline in omicsdm-server/pipelines/snakemake/all/ that orchestrates DESeq2, FGSEA, sc_normalisation, z_scoring, and gsva workflows in a single Docker container, as specified in the README requirements.

Pipeline Architecture

DESeq2 → FGSEA (significant_genesets.gmt) → z_scoring
                                          ↘
sc_normalisation ────────────────────────→ gsva

FGSEA extracts significant gene sets from bulk RNA-seq results, which then drive single-cell gene set scoring in parallel via z_scoring and gsva.

Implementation

• Snakefile: Defines workflow with proper dependencies between 12 rules (5 analysis stages, 5 data retrieval, 1 report, 1 orchestration)
• Dockerfile: Unified image with R 4.4.0 (Bioconductor) + Python 3.10 (scanpy/anndata) + Snakemake
• Source organization: Symlinks to individual pipeline scripts to avoid duplication; added fgsea_wrapper.R to generate GMT output
• Configuration: Per-pipeline config files; renv.lock for R dependencies; requirements.txt for Python packages
• Tooling: Makefile for build/run, docker-compose.yaml for orchestration

Snakemake Validation

$ snakemake --dry-run -j1
Job stats:
job                          count
-------------------------  -------
all                              1
deseq2_analysis                  1
fgsea_analysis                   1
sc_normalisation                 1
z_scoring                        1
gsva                             1
...
total                           12

All dependencies correctly resolved with FGSEA completion gating downstream gene set scoring.

Original prompt

Start implementation

[Chronological Review: The conversation began with the user requesting to perform the task stated in the README.md file located in the OmicsDM repository. The user then indicated a desire to start the implementation of the task, which suggests a focus on executing the instructions outlined in the README.]

[Intent Mapping:

• User Request 1: "Do the task stated in the README.md" - The user is looking to execute the instructions provided in the README file.
• User Request 2: "Start implementation" - The user is signaling readiness to begin the actual coding or implementation process based on the README.]

[Technical Inventory:

• Repository: OmicsDM
• Current Branch: bulkRNAseq_toyexample
• Default Branch: main
• File of Interest: /home/ivo/projects/bioinfo/cnag/repos/omicsdm/omicsdm-server/pipelines/snakemake/all/README.md]

[Code Archaeology:

• The README.md file is the focal point, but specific code changes or functions have not yet been discussed or modified in this conversation.]

[Progress Assessment:

• Completed Tasks: The user has identified the task in the README.md.
• Partially Complete Work: The user is in the process of starting the implementation of the task.]

[Context Validation: The conversation has established that the user is focused on executing the task outlined in the README.md file, which is critical for understanding the next steps.]

[Recent Commands Analysis:

• Last Agent Commands: The user requested to "Do the task stated in the README.md" and then "Start implementation."
• Tool Results Summary: No specific tool results were generated as the conversation has been focused on user requests rather than tool execution.
• Pre-Summary State: The agent was actively discussing the implementation of the task from the README.md when the token budget was exceeded.
• Operation Context: The commands executed were directly related to the user's goal of implementing the instructions from the README.md file.]

1. Conversation Overview: - Primary Objectives: - "Do the task stated in the README.md" - The user aims to execute the instructions in the README. - "Start implementation" - The user is ready to begin the coding process based on the README. - Session Context: The conversation has revolved around the user's intent to implement a task from the README.md file in the OmicsDM repository. - User Intent Evolution: The user has consistently focused on executing the task outlined in the README, moving from identifying the task to expressing readiness to implement it.

2. Technical Foundation:

• Repository: OmicsDM
• Current Branch: bulkRNAseq_toyexample
• Default Branch: main
• File of Interest: /home/ivo/projects/bioinfo/cnag/repos/omicsdm/omicsdm-server/pipelines/snakemake/all/README.md

3. Codebase Status:

• File Name: README.md
• Purpose: Provides instructions for tasks related to the OmicsDM project.
• Current State: The user is preparing to implement the tasks outlined in this file.
• Key Code Segments: Not specified yet as the implementation has not begun.
• Dependencies: The README serves as a guide for the implementation process.

4. Problem Resolution:

• Issues Encountered: None reported yet.
• Solutions Implemented: None yet, as the user is still in the preparatory phase.
• Debugging Context: Not applicable at this stage.
• Lessons Learned: No insights have been gathered yet.

5. Progress Tracking:

• Completed Tasks: The user has identified the task in the README.md.
• Partially Complete Work: The user is preparing to start the implementation.
• Validated Outcomes: None confirmed yet.

6. Active Work State:

• Current Focus: The user is focused on implementing the task from the README.md.
• Recent Context: The user has expressed readiness to start the implementation.
• Working Code: No specific code has been discussed or modified yet.
• Immediate Context: The user is preparing to execute the instructions from the README.md.

7. Recent Operations:

• Last Agent Commands: "Do the task stated in the README.md" and "Start implementation."
• Tool Results Summary: No tool results were generated as the conversation has been focused on user requests.
• Pre-Summary State: The agent was discussing the implementation of the task from the README.md.
• Operation Context: The commands executed were directly related to the user's goal of implementing the instructions from the README.md file.

8. Continuation Plan:

• Pending Task 1: Implement the task as stated in the README.md.
• Pending Task 2: Review the README.md for specific instructions to follow.
• Priority Information: The immediate next step is to begin the implementation process.
• Next Action: The user should start coding based on the instructions in the README.md.

Created from VS Code.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.