gistfile1.txt

Running three different comparisons (see `scripts/rtg*`) using the _filtered_ SNP data this time around. First, the gold standard (5/9 samples) against each filtered call set, `rtg vcfeval -b Shared.vcf.gz -c foo.vcf.gz -t /cm/shared/apps/bcbio/20150720-devel/data/genomes/Hsapiens/GRCh37/rtg/GRCh37.sdf -o eval`:


File  							        True-pos  False-pos  False-neg  Precision  Sensitivity  F-measure
filtered.Fresh_4706				  3338450      54119     339450     0.9840       0.9077     0.9443
filtered.Fresh_4707				  3409349     154908     268551     0.9565       0.9270     0.9415
filtered.Fresh_Normal_4708	3464245     190615     213655     0.9478       0.9419     0.9449
filtered.UMFIX_4085				  3134462     145638     543438     0.9556       0.8522     0.9010
filtered.UMFIX_4090				  3101645     105139     576255     0.9672       0.8433     0.9010
filtered.UMFIX_Normal_4088	3268466     157860     409434     0.9539       0.8887     0.9201
filtered.NBF_4087				    3217841     109510     460059     0.9671       0.8749     0.9187
filtered.NBF_4091				    3090858     105003     587042     0.9671       0.8404     0.8993
filtered.NBF_Normal_4089		2932151     132203     745749     0.9569       0.7972     0.8698

This shows the previously mentioned bias for 4706. Trying to fix that with the `--squash-ploidy` option that ignores het/hom differences:

File  							        True-pos  False-pos  False-neg  Precision  Sensitivity  F-measure
filtered.Fresh_4706				  3338450      54119     339450     0.9840       0.9077     0.9443
filtered.Fresh_4707				  3440339     123918     237561     0.9652       0.9354     0.9501
filtered.Fresh_Normal_4708	3493079     161781     184821     0.9557       0.9497     0.9527
filtered.UMFIX_4085				  3167121     112979     510779     0.9656       0.8611     0.9104
filtered.UMFIX_4090				  3133905      72879     543995     0.9773       0.8521     0.9104
filtered.UMFIX_Normal_4088	3298795     127531     379105     0.9628       0.8969     0.9287
filtered.NBF_4087				    3249176      78175     428724     0.9765       0.8834     0.9276
filtered.NBF_4091				    3122006      73855     555894     0.9769       0.8489     0.9084
filtered.NBF_Normal_4089		2960236     104118     717664     0.9660       0.8049     0.8781

This improves results across the board, but 4706 still has the lowest false positive rate. Wonder if there's another bias we introduce by picking the genotype from the first sample. Repeat with the calls limited to the GiaB regions:

File  							        True-pos  False-pos  False-neg  Precision  Sensitivity  F-measure
filtered.Fresh_4706				  2457182      30451     231415     0.9878       0.9139     0.9494
filtered.Fresh_4707				  2540910      65139     147687     0.9750       0.9451     0.9598
filtered.Fresh_Normal_4708	2590017      90894      98580     0.9661       0.9633     0.9647
filtered.UMFIX_4085				  2323935      53437     364662     0.9775       0.8644     0.9175
filtered.UMFIX_4090				  2309402      34995     379195     0.9851       0.8590     0.9177
filtered.UMFIX_Normal_4088	2438734      70336     249863     0.9720       0.9071     0.9384
filtered.NBF_4087				    2402207      39417     286390     0.9839       0.8935     0.9365
filtered.NBF_4091				    2310635      35588     377962     0.9848       0.8594     0.9179
filtered.NBF_Normal_4089		2191893      53634     496704     0.9761       0.8153     0.8885

Good news: while the fresh frozen samples look better across the board there's not a huge difference between UMFIX and NBF. We also have enough FP and FN to extract signal from the three fixation subgroups.