Skip to content

Instantly share code, notes, and snippets.

View craffel's full-sized avatar

Colin Raffel craffel

View GitHub Profile
@craffel
craffel / Count github LOC.ipynb
Created April 24, 2016 22:24
Count your total contributions on GitHub of a certain filetype (a hack)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@craffel
craffel / gist:76924fd211720412c19327cfb8f0f5e2
Created June 21, 2017 19:20
Vim regex for replacing all commas not followed by space with commas followed by space
:%s/,\([^ ]\)/, \1/g
@craffel
craffel / Tutorial.ipynb
Created October 21, 2017 04:52
pretty_midi tutorial (draft)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@craffel
craffel / nqyn-dev.tsv
Created August 20, 2020 12:40
Yes/no questions in Natural Questions
dominant alleles are always the most common allele in a population NO
is there a basketball team in las vegas YES
extinction can be the result of habitat fragmentation YES
is area code 808 a toll free number NO
does a south african passport need a visa for singapore NO
does newman's own really give all profits to charity YES
is the son of the beast still at kings island NO
all types of joints allow for the same range of motion NO
do ghanaians need visa to go to singapore NO
is the right to life liberty and the pursuit of happiness in the constitution NO
@craffel
craffel / eval_results.csv
Created February 18, 2025 15:26
Eval results from fineweb ablations
We can make this file beautiful and searchable if this error is corrected: It looks like row 4 should actually have 21 columns, instead of 3 in line 3.
runname,steps,agg_score,commonsense_qa/acc,commonsense_qa/acc_norm,hellaswag/acc,hellaswag/acc_norm,openbookqa/acc,openbookqa/acc_norm,piqa/acc,piqa/acc_norm,siqa/acc,siqa/acc_norm,winogrande/acc,winogrande/acc_norm,sciq/acc,sciq/acc_norm,arc/acc,arc/acc_norm,mmlu/acc,mmlu/acc_norm
C4,0,0.3308933284133672,0.1860000044107437,0.2329999953508377,0.2720000147819519,0.257999986410141,0.1659999936819076,0.2860000133514404,0.5419999957084656,0.5099999904632568,0.367000013589859,0.3619999885559082,0.515999972820282,0.4970000088214874,0.2080000042915344,0.202000007033348,0.2195000052452087,0.2509999871253967,0.2302941530942917,0.2501466572284698
C4,1000,0.3551123738288879,0.2290000021457672,0.2599999904632568,0.2860000133514404,0.2879999876022339,0.1280000060796737,0.25,0.6140000224113464,0.6190000176429749,0.3510000109672546,0.4040000140666961,0.5189999938011169,0.4760000109672546,0.5649999976158142,0.5180000066757202,0.2680000066757202,0.293500006198883,0.2389510571956634,0.2503989636898041
C4,2000,0.378435287624597