Skip to content

Instantly share code, notes, and snippets.

@tgirke
tgirke / processFileInChunks.R
Last active December 25, 2018 22:44
Streaming through large tabular files in R
################################################
## Streaming through large tabular files in R ##
################################################
## Author: Thomas Girke
## Last update: 21-Dec-2018
## Utility: the function 'processFileInLineChunks' streams through a file in
## batches of lines and applies to each imported line batch a function assigned
## to the 'myFct' argument. The number of lines processed in each iteration can
## be defined under the 'n_rows' argument. As importer functions, both 'fread'
## Arranging plots with patchwork. For details see this manual page: https://ggplot2-book.org/arranging-plots.html
## Load libraries and create some sample data
library(ggplot2); library(reshape2); library(patchwork)
iris_mean <- aggregate(iris[,1:4], by=list(Species=iris$Species), FUN=mean)
df_mean <- melt(iris_mean, id.vars=c("Species"), variable.name = "Samples", value.name="Values")
## Create lots of plots
p1 <- ggplot(cbind(iris, Samples="Sepal.Length"), aes(Species, Sepal.Length)) + geom_boxplot(aes(fill = Species)) + ylim(4, 8) + theme(axis.title.x=element_blank(), axis.title.y=element_blank(), legend.position = "none") + facet_grid(rows = ~Samples) + ggtitle("(A)") + theme(plot.title = element_text(hjust=-0.3))
p2 <- ggplot(df_mean[df_mean$Samples=="Sepal.Width",], aes(Species, Values)) + geom_bar(position="dodge", aes(fill = Species), stat="identity") + yli