Benchmarks for http://stackoverflow.com/questions/31206735/pivoting-a-large-data-set

SO31206735.R

library(data.table)
library(dplyr)
library(tidyr)
library(stringi)
library(microbenchmark)

set.seed(1)
ndim <- 10000
ndate <- 1200

mydf <- setDF(CJ(Dimension = sequence(ndim), 
                 Date = stri_rand_strings(ndate, 5))[
                   , Metric := sample(ndate, ndim, TRUE)
                 ])


funTidy <- function(indf) {
  indf %>% 
    spread(Date, Metric, fill = 0)
}

funDT <- function(indf) {
  dcast.data.table(as.data.table(indf), Dimension ~ Date, value.var = "Metric")
}

funBase <- function(indf) {
  xtabs(Metric ~ Dimension + Date, indf)
}

# microbenchmark(funTidy(mydf), funDT(mydf), funBase(mydf))

system.time(funTidy(mydf))
system.time(funDT(mydf))
system.time(funBase(mydf))

Results on my system:

system.time(funTidy(mydf))
#    user  system elapsed 
#   6.376   0.016   6.463 
system.time(funDT(mydf))
#    user  system elapsed 
#   1.800   0.004   1.828 
system.time(funBase(mydf))
#    user  system elapsed 
#  58.048   0.228  61.468 

Session Info:

sessionInfo()
## R version 3.2.0 (2015-04-16)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 15.04
## 
## locale:
##  [1] LC_CTYPE=en_IN.UTF-8       LC_NUMERIC=C               LC_TIME=en_IN.UTF-8        LC_COLLATE=en_IN.UTF-8    
##  [5] LC_MONETARY=en_IN.UTF-8    LC_MESSAGES=en_IN.UTF-8    LC_PAPER=en_IN.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_IN.UTF-8 LC_IDENTIFICATION=C       
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] stringi_0.4-1        microbenchmark_1.4-2 tidyr_0.2.0          dplyr_0.4.1          data.table_1.9.4    
## [6] overflow_0.2-2       gtools_3.5.0        
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_0.11.6      magrittr_1.5     MASS_7.3-40      munsell_0.4.2    colorspace_1.2-6 R6_2.0.1        
##  [7] stringr_1.0.0    plyr_1.8.3       tools_3.2.0      parallel_3.2.0   grid_3.2.0       gtable_0.1.2    
## [13] DBI_0.3.1        lazyeval_0.1.10  assertthat_0.1   digest_0.6.8     reshape2_1.4.1   ggplot2_1.0.1   
## [19] scales_0.2.5     chron_2.3-45     proto_0.3-10    

Just a difference. My test on SO were: two objects (a data frame and a data table) and not one converted as.data.table within the function call. Anyway now data table is faster.

:: > system.time(funTidy(mydf))
user system elapsed
17.466 1.792 19.046

system.time(funDT(mydf))
user system elapsed
4.003 0.467 4.415
system.time(funBase(mydf))
user system elapsed
105.175 4.709 114.969

sessionInfo()
R version 3.2.1 (2015-06-18)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.10.4 (Yosemite)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] microbenchmark_1.4-2 stringi_0.5-5 tidyr_0.2.0 dplyr_0.4.2
[5] data.table_1.9.4

loaded via a namespace (and not attached):
[1] Rcpp_0.11.6 digest_0.6.8 assertthat_0.1 MASS_7.3-41 grid_3.2.1
[6] chron_2.3-47 plyr_1.8.3 R6_2.0.1 gtable_0.1.2 DBI_0.3.1
[11] magrittr_1.5 scales_0.2.5 ggplot2_1.0.1 reshape2_1.4.1 proto_0.3-10
[16] tools_3.2.1 stringr_1.0.0 munsell_0.4.2 parallel_3.2.1 colorspace_1.2-6

::