Skip to content

Instantly share code, notes, and snippets.

@aviks
Created October 31, 2017 01:24
Show Gist options
  • Save aviks/2a06cb97e20292841f8980ee18502510 to your computer and use it in GitHub Desktop.
Save aviks/2a06cb97e20292841f8980ee18502510 to your computer and use it in GitHub Desktop.
using DataValues
julia> for n=1:25 [29/1811]
a=DataValueArray(rand(2^n))
@time map(x->isnull(x) ? 0.0 : x, a);
end
0.118944 seconds (24.93 k allocations: 1.254 MiB)
0.000013 seconds (8 allocations: 368 bytes)
0.000007 seconds (8 allocations: 400 bytes)
0.000007 seconds (8 allocations: 480 bytes)
0.000007 seconds (8 allocations: 688 bytes)
0.000007 seconds (8 allocations: 944 bytes)
0.000009 seconds (8 allocations: 1.516 KiB)
0.000012 seconds (8 allocations: 2.672 KiB)
0.000016 seconds (8 allocations: 4.891 KiB)
0.000024 seconds (8 allocations: 9.422 KiB)
0.000040 seconds (8 allocations: 18.469 KiB)
0.000074 seconds (9 allocations: 36.422 KiB)
0.000141 seconds (9 allocations: 72.422 KiB)
0.000281 seconds (10 allocations: 144.375 KiB)
0.000544 seconds (10 allocations: 288.375 KiB)
0.001150 seconds (10 allocations: 576.375 KiB)
0.002345 seconds (10 allocations: 1.125 MiB)
0.004311 seconds (10 allocations: 2.250 MiB)
0.008569 seconds (10 allocations: 4.500 MiB)
0.017027 seconds (10 allocations: 9.000 MiB)
0.034683 seconds (10 allocations: 18.000 MiB)
0.083858 seconds (10 allocations: 36.000 MiB, 1.19% gc time)
0.284225 seconds (10 allocations: 72.000 MiB, 41.54% gc time)
0.450484 seconds (10 allocations: 144.000 MiB, 26.54% gc time)
0.817855 seconds (10 allocations: 288.000 MiB, 17.41% gc time)
#But add an NA to the array, and it goes haywire!!
julia> for n=1:25
a=DataValueArray(rand(2^n)); a[2]=DataValues.NA
@time map(x->isnull(x) ? 0.0 : x, a);
end
0.149185 seconds (28.08 k allocations: 1.411 MiB)
0.000059 seconds (41 allocations: 1.844 KiB)
0.000060 seconds (107 allocations: 4.844 KiB)
0.000117 seconds (287 allocations: 13.344 KiB)
0.000226 seconds (839 allocations: 41.594 KiB)
0.000473 seconds (2.71 k allocations: 131.594 KiB)
0.001227 seconds (9.53 k allocations: 467.594 KiB)
0.002899 seconds (35.45 k allocations: 1.703 MiB)
0.009148 seconds (136.44 k allocations: 6.515 MiB)
0.157929 seconds (536.57 k allocations: 25.586 MiB, 82.92% gc time)
0.147648 seconds (2.12 M allocations: 101.289 MiB, 44.82% gc time)
0.898151 seconds (8.45 M allocations: 402.414 MiB, 70.65% gc time)
5.326090 seconds (33.67 M allocations: 1.567 GiB, 82.23% gc time)
38.775815 seconds (134.47 M allocations: 6.259 GiB, 91.69% gc time)
<Ctrl-c after about an hour!>
InterruptException:
Stacktrace:
[1] similar(::DataValues.DataValueArray{Any,1}, ::Type{Any}) at ./abstractarray.jl:520
[2] collect_to!(::DataValues.DataValueArray{Any,1}, ::Base.Generator{DataValues.DataValueArray{Float64,1},##25#26}, ::Int64, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:481
[3] collect_to!(::DataValues.DataValueArray{Any,1}, ::Base.Generator{DataValues.DataValueArray{Float64,1},##25#26}, ::Int64, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:484 (repeats 1381 times)
[4] collect_to_with_first!(::DataValues.DataValueArray{Float64,1}, ::Float64, ::Base.Generator{DataValues.DataValueArray{Float64,1},##25#26}, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:461
[5] _collect(::DataValues.DataValueArray{Float64,1}, ::Base.Generator{DataValues.DataValueArray{Float64,1},##25#26}, ::Base.EltypeUnknown, ::Base.HasShape) at ./array.jl:455
[6] map(::Function, ::DataValues.DataValueArray{Float64,1}) at ./abstractarray.jl:1865
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment