Skip to content

Instantly share code, notes, and snippets.

@alexpreynolds

alexpreynolds/gist:af2d1b4294cbbede9a49b2e9eb86528d

Last active September 22, 2020 22:38
Show Gist options
  • Save alexpreynolds/af2d1b4294cbbede9a49b2e9eb86528d to your computer and use it in GitHub Desktop.
Save alexpreynolds/af2d1b4294cbbede9a49b2e9eb86528d to your computer and use it in GitHub Desktop.
Download ZIP
bigWig to Starch compression ratio and `unstarch --list-json`/`unstarch <chr>` memory usage
Raw
gistfile1.txt
BigWig to BedGraph
$ wget -qO- https://www.encodeproject.org/files/ENCFF978IHV/@@download/ENCFF978IHV.bigWig > ENCFF978IHV.bigWig
$ bigWigToBedGraph ENCFF978IHV.bigWig ENCFF978IHV.bigWig.bg
BedGraph to Starch
$ starch --omit-signature ENCFF978IHV.bigWig.bg > ENCFF978IHV.bigWig.bg.starch
Compression ratio
$ STARCH_FILE_SIZE=`wc -c < ENCFF978IHV.bigWig.bg.starch`
$ BEDGRAPH_FILE_SIZE=`wc -c < ENCFF978IHV.bigWig.bg`
$ calc "${STARCH_FILE_SIZE}/${BEDGRAPH_FILE_SIZE}"
~0.02330927077202778608
Memory usage (metadata output)
$ gtime -v unstarch --list-json ~/Downloads/ENCFF978IHV.bigWig.bg.starch 1>/dev/null
...
Maximum resident set size (kbytes): 1476
...
Memory usage (per-chromosome output, chr1)
$ gtime -v unstarch chr1 ~/Downloads/ENCFF978IHV.bigWig.bg.starch 1>/dev/null
...
Maximum resident set size (kbytes): 5124
...
Memory usage (per-chromosome output, chr2)
$ gtime -v unstarch chr2 ~/Downloads/ENCFF978IHV.bigWig.bg.starch 1>/dev/null
...
Maximum resident set size (kbytes): 5140
...
@alexpreynolds
Copy link
Author

alexpreynolds commented Sep 22, 2020 •
edited
Loading

I am running BEDOPS v2.4.39 (typical build) on Mac OS X 10.15.6. I used Homebrew to install GNU time (gtime).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment