small number of reads / no aberrant reads triggers exception in chrom_diff.r

[GWarsow]

chrom_diff.r is called in bwaMemSortSlim.sh via

AlignmentAndQCWorkflows/resources/analysisTools/qcPipeline/bwaMemSortSlim.sh

Line 276 in 2701b22

${RSCRIPT_BINARY} ${TOOL_PLOT_DIFFCHROM} -i "$FILENAME_DIFFCHROM_MATRIX" -s "$FILENAME_DIFFCHROM_STATISTICS" -o "${FILENAME_DIFFCHROM_PLOT}_temp" && mv ${FILENAME_DIFFCHROM_PLOT}_temp ${FILENAME_DIFFCHROM_PLOT} || ( echo "Error from chrom_diff.r" && exit 23)

The paramter -i "$FILENAME_DIFFCHROM_MATRIX" tells the R script to use the content of the file FILENAME_DIFFCHROM_MATRIX (*_DiffChroms.txt), which is generated by TOOL_COMBINED_BAM_ANALYSIS (flags_isizes_PEaberrations.pl):

AlignmentAndQCWorkflows/resources/analysisTools/qcPipeline/bwaMemSortSlim.sh

Line 130 in 2701b22

(${PERL_BINARY} ${TOOL_COMBINED_BAM_ANALYSIS} -i ${NP_COMBINEDANALYSIS_IN} -a ${FILENAME_DIFFCHROM_MATRIX}.tmp -c ${CHROM_SIZES_FILE} -b ${FILENAME_ISIZES_MATRIX}.tmp -f ${FILENAME_EXTENDED_FLAGSTATS}.tmp -m ${FILENAME_ISIZES_STATISTICS}.tmp -o ${FILENAME_DIFFCHROM_STATISTICS}.tmp -p ${INSERT_SIZE_LIMIT} ) & procIDCBA=$!

FILENAME_DIFFCHROM_MATRIX shall contain a matrix. However, in flags_isizes_PEaberrations.pl, if no aberrant read could be identified (which can happen easily if the number of reads is small), only "NA" is written instead of the matrix.

This causes an exception, when chrom_diff.r tries to use the content of FILENAME_DIFFCHROM_MATRIX to do

AlignmentAndQCWorkflows/resources/analysisTools/qcPipeline/chrom_diff.r

Lines 35 to 37 in 2701b22

	for(i in 1:nrow(data)){
	v[i] <- sum(data[i,])
	}

(data contains the content of FILENAME_DIFFCHROM_MATRIX)

The exception which is thrown is:

Error in FUN(X[[i]], ...) : 
  only defined on a data frame with all numeric variables

[GWarsow]

Context / example case:

/omics/odcf/otp_data/003/05/9b/a4b5-2530-4e0a-a67a-b4c7c0d46aa0/roddyExecutionStore/exec_241203_141239920_icgcdata_analysis/r241203_141239920_OE0538_DO-0001_mmus_DT_3_alignAndPairSlim.o34053968

the log says:

no aberrant paired reads found, single end reads?

this finally leads to

print MA "NA\n";

with MA being the _DiffChroms.txt file, which is input for the R script chrom_diff.r

In this script, the following error is thrown

Error in FUN(X[[i]], ...) : 
  only defined on a data frame with all numeric variables

The working hypothesis is that this no aberrant paired reads found situation occurs due to the low number of reads in the fastq files (366 sequences):

AS-758433-LR-60460_R*.fastq.gz

[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 366 sequences (36966 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 149, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (125, 180, 256)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 518)
[M::mem_pestat] mean and std.dev: (195.47, 88.89)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 649)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 366 reads in 0.106 CPU sec, 0.020 real sec
[samopen] SAM header is present: 67 sequences.

There were other jobs that were comparable but seemed to work correctly:

AS-758433-LR-60420_R*.fastq.gz

[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 430 sequences (43430 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 162, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (124, 179, 246)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 490)
[M::mem_pestat] mean and std.dev: (188.64, 82.53)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 612)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 430 reads in 0.153 CPU sec, 0.028 real sec
[samopen] SAM header is present: 67 sequences.

Here, we have 1 (one) aberrant read which leads to correctly filled matrix file *_DiffChroms.txt, with only zeroes in the cells - but this is at least parsable for chrom_diff.r.

related pages in OTP:
https://otp.dkfz.de/otp/clusterJobDetail/showLog/167995009?workflowRun.id=167993523
https://otp.dkfz.de/otp/workflowRunDetails/index/167993523