vcfdist v3.0.0 by TimD1 · Pull Request #37 · TimD1/vcfdist

TimD1 · 2025-04-06T14:47:44Z

Main Goals for v3.0.0

Avoid memory/runtime explosion in dense variant regions
Support large SVs (ideally raise limit from 10kb to 100kb)
Per-variant phasing analysis
Genotyping error summary

Required Deliverables

Code Deliverables

Runtime/Accuracy Investigation

maximum supercluster size
bi-wfa clustering iterations
adding clustering heuristics
different number of retries with FNs

Analysis Deliverables

regenerate accuracy plots from previous paper, showing v3 is better
regenerate runtime/memory plots from previous paper

Documentation Deliverables

update documentation/wiki
write blogpost summary of improvements
write BlueSky/LinkedIn post

Minor TODOs

update memory calculation?
update thread limiting logic (thread2 unused)

Limitations and Assumptions

Phase groups need all variants to be adjacent. If there is an unphased variant in the middle of a phase group, it will be split up. (for example, the test VCF in ./run doesn’t phase heterozygous alternate variants)

Nice to Have

Related Projects

add complex variant normalization as bcftools extension
add nf-core module for vcfdist
add multiqc plugin for vcfdist
sandbox.bio tutorial for vcfdist

Potential Improvements

FN retrying should fix FN SVs "swallowing" TP SNPs, but what about TP SVs "swallowing" FN/FP SNPs?
biWFA clustering requires variants to be included. What about graph-based alignment here?

Efficiency Optimizations

wf_swg_align() DP matrix only needs offsets for N most recent rows
shift from Dijkstra to A*?
no traceback for retries only calculating distance

Punting to v3.1

expand support to SV tags with precise breakpoints
evaluate unphased variants
retain all INFO/FORMAT fields in original VCFs
- make summary.vcf output fields optional
add more thorough integration tests
- bacterial genome (haploid)
- VCF against itself (change normalization?)
- empty truth or query VCF
- mix of upper/lower case
feedback on desired outputs (fp.vcf, fn.vcf, tp.vcf?)
- output dropped/unevaluated variants to VCF?

Runtime Analysis

Helium, 16 threads, 64GB RAM, max variant 1000

PAV

SNP F1	INDEL F1	SV F1	cluster	prec-recall	max ED	biWFA	max retries
0.973031	0.963212	0.956453	1922.912	22400.579	1000	4	5
0.972793	0.962972	0.948846	1908.611	9519.489	100	4	5
0.972688	0.962791	0.948865	593.576	8812.550	100	1	5
0.972690	0.962811	0.948748	586.249	5794.202	100	1	3
0.972654	0.962774	0.947229	584.666	2887.851	100	1	1
*	*	*	584.441	260.692	100	1	0

HPRC

SNP F1	INDEL F1	SV F1	cluster	prec-recall	max ED	biWFA	max retries
0.998519	0.984859	0.995799	2055.939	2105.901	1000	4	5
0.998497	0.984855	0.993183	2036.129	1226.166	100	4	5
0.998498	0.984847	0.993182	578.310	1116.628	100	1	5
0.998500	0.984848	0.993230	581.234	552.833	100	1	3
0.998496	0.984838	0.992879	584.549	317.261	100	1	1
*	*	*	583.830	106.741	100	1	0

GIAB-TR

SNP F1	INDEL F1	SV F1	cluster	prec-recall	max ED	biWFA	max retries
0.982936	0.982728	0.969777	1998.548	20667.432	1000	4	5
0.982458	0.982524	0.964042	2004.119	11258.847	100	4	5
0.979404	0.981241	0.962295	571.643	11466.438	100	1	5
0.979451	0.981298	0.962706	565.778	7439.718	100	1	3
0.979339	0.981249	0.961703	570.376	3711.576	100	1	1
*	*	*	569.330	313.688	100	1	0

- no real changes to analysis, just minor visual updates

- simplified, no more plotting SNP, INDEL, or LARGE - stats written to json file, added script for calc

- changed to at most 2 threads per supercluster - started high-level reorganization

- this occurs after clustering but before superclustering - this will allow graph generation, with genotype information - introduced add_callset_vars() - added variantData->nc, moved ctg_variants to callset_vars

- updated Graph definition/structure with additional info

- always select INS first if multiple variants occur at same position

- main `calc_prec_recall_aln()` function is complete - added +1 to last query graph node length - added full alignment graph printing for debugging - truth string generation checks if variant occurs on hap

- truth sequence is now a path through a graph - this should enable much simple logic for parsing sync groups etc

- incorrect, but everything completes - still need to uncomment and update initial phaseset printing

- FP counting is still wrong: if calc_gt is 0|0, it's a FP

- fix: end WFA early if max score is reached - change: only consider redoing alignment if FN is an INDEL (not SUB) - fix: don't parse WFA path if alignment didn't finish - feat: removed backup ordinary alignment

- this is expected if the last variant is a deletion and the cluster expands to the end of the reference

TimD1 and others added 30 commits March 22, 2024 14:13

Updated phab_cm

ee3eb60

- no real changes to analysis, just minor visual updates

Updated FNR/FDR plot for small_sv_all

3dc8fda

- simplified, no more plotting SNP, INDEL, or LARGE - stats written to json file, added script for calc

Shifted vs_prior_work to FDR, updated plot visuals

329f542

Ignoring JSON.

ec324c6

Removed obsolete reverse_ptrs_strs()

67cbbfd

Removed edit distance from vcfdist entirely.

9e95ff5

Started shifting to graph alignment

a7a111d

- changed to at most 2 threads per supercluster - started high-level reorganization

Minor plot updates.

fd54fe3

Removed edit.*

84233a8

Removed edit.* from Makefile.

69be7c2

Variant information is merged across haplotypes

b9e95af

- this occurs after clustering but before superclustering - this will allow graph generation, with genotype information - introduced add_callset_vars() - added variantData->nc, moved ctg_variants to callset_vars

Started making graph data structure.

02a5f57

Added graph initialization

b462854

- updated Graph definition/structure with additional info

Added htslib dependency installation.

c5c8266

Fixed edge case for cluster merging

56063a1

- always select INS first if multiple variants occur at same position

Added forward-pass alignment

da3ec09

- main `calc_prec_recall_aln()` function is complete - added +1 to last query graph node length - added full alignment graph printing for debugging - truth string generation checks if variant occurs on hap

Updated graph definition to make truth a graph.

5d95876

Forward alignment uses new algorithm

6c5cd1d

- truth sequence is now a path through a graph - this should enable much simple logic for parsing sync groups etc

Added precision-recall calculation.

b3e0581

Updated phasing and printing

cc4a810

- incorrect, but everything completes - still need to uncomment and update initial phaseset printing

Continued bug fixes.

c32a338

- FP counting is still wrong: if calc_gt is 0|0, it's a FP

fix: included bugfixes from v2.5.3

5ac1532

cleanup: removed obsolete unreachable functions

232036e

uncommented and fixed initial per-VCF NG50 calculation

0021eb3

docs: added v2.5.3 to cache

6540a36

feat: added Google Test framework

674a9b4

cleanup: removed obsolete constants

a5bf84a

ci: added tests

6c599ae

ci: updated test.yml

25a6d4c

wip: updating credit calc

6eb7fd2

TimD1 added 21 commits April 6, 2025 11:39

fix: remove git-aware from pytest

11efcb4

fix: WFA error when variants are adjacent to contig ends

e50a05c

feat: WFA only precision/recall alignment!

099c255

- fix: end WFA early if max score is reached - change: only consider redoing alignment if FN is an INDEL (not SUB) - fix: don't parse WFA path if alignment didn't finish - feat: removed backup ordinary alignment

fix: calculate WFA RAM usage

e4ec98c

perf: clear WFA matrices ASAP

a0ebe8e

feat: added memory efficient ED gWFA (w/o backtracking)

6673744

fix: skip FN redo if type is sub, not length 1

3fd06be

feat: pack WFA pointers

d350ca1

wip: draft condensing alignment memory, segfaults

df1fa44

fix: if vs else if, continue if unallocated

0e8f67b

feat: revert to single-pass evaluation

159d6bc

fix: updated timer printing

1797c35

fix: WFA pointers now packed into uint64_t

c062b8c

feat: revert from gWFA -> Dijkstra

e48a43f

feat: multi-match in Dijkstra

a87b7a7

perf: remove backtrack option from wf_ed() and wf_swg_aln()

d66f155

chore: remove obsolete calc_cig_swg_score() and extract_errors()

ed4888b

chore: removed cluster and variant merging

1ae14b9

feat: added --max-supercluster-size from v2

41cc0a0

fix: rounding error in NG50 calculation

d7313ec

fix: replaced obsolete CIGAR score test with NG50 test

26d7aa4

TimD1 self-assigned this Jun 22, 2025

TimD1 added 8 commits June 21, 2025 23:04

chore(test): reorganized unit test src

2be7a41

chore: update citation

31a8f78

chore: remove obsolete print_cigar()

c945b34

style: rename c -> ci for callset index

60925cd

fix: increment nc (number of clusters) when splitting cluster

c23dc2d

feat: better genotype error summary format

8afc267

fix: force VCF REF/ALT and ref FASTA to use upper-case

946646b

fix: reaching end of query before end of reference isn't an error

1588ed0

- this is expected if the last variant is a deletion and the cluster expands to the end of the reference

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vcfdist v3.0.0#37

vcfdist v3.0.0#37
TimD1 wants to merge 131 commits intomasterfrom
dev

TimD1 commented Apr 6, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

TimD1 commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Main Goals for v3.0.0

Required Deliverables

Limitations and Assumptions

Nice to Have

Runtime Analysis

PAV

HPRC

GIAB-TR

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

TimD1 commented Apr 6, 2025 •

edited

Loading