Skip to content

working version 1.0#1

Closed
manascripts wants to merge 11 commits into
mainfrom
dev
Closed

working version 1.0#1
manascripts wants to merge 11 commits into
mainfrom
dev

Conversation

@manascripts
Copy link
Copy Markdown
Owner

This pull request updates resource allocations and input schema for the pipeline to improve sample handling, process reliability, and computational efficiency. The most significant changes are the overhaul of the sample input schema, increased resource requests for several modules, and refined process retry logic. Additionally, output naming conventions and memory scaling have been adjusted for consistency and performance.

Input Schema Changes

  • The sample input schema in assets/schema_input.json is restructured: sample and FastQ file fields are replaced with group_id, sample_id, sample_type, and bam_path, making the schema compatible with BAM-based workflows and supporting case/control sample types. Required fields are updated accordingly.

Resource Allocation Updates

  • Many processes in conf/modules.config now request higher CPU and memory resources, especially for structural variant callers (SNIFFLES, CUTESV, SEVERUS_WITH_CONTROL, SEVERUS_NO_CONTROL, JASMINESV_SAMPLE, JASMINESV_COHORT, SVDB_QUERY_SAMPLE, SVDB_QUERY_COHORT). Memory allocations now often scale with task.attempt for better retry handling. [1] [2] [3] [4] [5] [6] [7] [8]

Process Reliability and Retry Logic

  • The default maximum number of retries for failed processes is increased from 1 to 2 in conf/base.config, and the process timeout is re-enabled.

Output Naming and Metadata Consistency

  • Output file naming in annotation steps now uses group_id instead of sample, aligning with the new schema and improving metadata consistency. [1] [2]

Memory Scaling and Miscellaneous Adjustments

  • Memory allocations for header renaming, annotation, summary, and utility steps are updated to scale with task.attempt, and several intermediate steps now explicitly set resource requirements. Some process labels have time settings commented out for flexibility. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

@manascripts manascripts deleted the dev branch September 29, 2025 13:40
@manascripts manascripts restored the dev branch September 29, 2025 13:40
@manascripts manascripts reopened this Sep 29, 2025
@manascripts manascripts closed this Oct 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant