Skip to content

Instantly share code, notes, and snippets.

View mwiewior's full-sized avatar

Marek Wiewiórka mwiewior

View GitHub Profile
scala> carbon.sql("""select * from dwh_dev2.fact_orc where f_variant_name='chr1:896764_G/C' and f_sample_id=41447""").show
+-----------+--------+------+---------------+--------------+---------+---------------+-----+------+-----+-----+-------------+-------------+----------+
|f_sample_id|f_geo_id|f_d_id|f_ensembl_gp_id|f_refseq_gp_id|   f_v_id| f_variant_name|f_chr| f_pos|f_ref|f_alt|f_alter_depth|f_total_depth|f_genotype|
+-----------+--------+------+---------------+--------------+---------+---------------+-----+------+-----+-----+-------------+-------------+----------+
|      41447|     384|     0|            213|          null|377052304|chr1:896764_G/C| chr1|896764|    G|    C|           13|           24|       0/1|
|      41447|     384|     0|           null|        317151|377052304|chr1:896764_G/C| chr1|896764|    G|    C|           13|           24|       0/1|
|      41447|     384|     0|           null|         64563|377052304|chr1:896764_G/C| chr1|896764|    G|    C|           13|           2
@mwiewior
mwiewior / qualcounts.scala
Created January 20, 2016 07:36
Counting quals
.mapPartitions {
partIterator =>
//val qualStrings = partIterator.toSeq
val maxReadQual=150
val maxReadLength=150
var pIndex = 0
val posQualArray = Array.fill[Int](maxReadLength,maxReadQual)(0) //((pos,qualVal),counter)
for (seq <- partIterator) {
var qIndex = 0
for(q<-seq.getQual){

Phoenix/Spark demo

Option 1: prebuilt VM

There is a prebuilt Centos 6.5 VM with the below components installed:

  • HDP 2.3.0.0-1754
  • Spark 1.3.1