Skip to content

Instantly share code, notes, and snippets.

@den-crane
Created December 27, 2019 17:11
Show Gist options
  • Save den-crane/329aa595c5a2f96fa36e009d4b17a382 to your computer and use it in GitHub Desktop.
Save den-crane/329aa595c5a2f96fa36e009d4b17a382 to your computer and use it in GitHub Desktop.
json_load_test
clickhouse-client -q "drop table json"
clickhouse-client -q "select arrayStringConcat(arrayMap(x->toString (cityHash64(x)) , range(1000)),',\"''') from numbers(100000);" >test.json
clickhouse-client -q "create table json(j String) Engine=Memory"
time clickhouse-client --format_csv_allow_single_quotes=0 --format_csv_allow_double_quotes=0 --format_csv_delimiter=$'\x0B' -q 'insert into json format CSV' <test.json
19.13.6.51
real 0m2.069s user 0m0.520s sys 0m0.746s
real 0m2.043s user 0m0.511s sys 0m0.722s
19.17.6.36
real 0m2.011s user 0m0.549s sys 0m0.625s
real 0m2.027s user 0m0.554s sys 0m0.654s
19.19.1.1957
real 0m1.473s user 0m2.268s sys 0m0.583s
real 0m1.405s user 0m2.155s sys 0m0.525s
@SaltTan
Copy link

SaltTan commented Dec 27, 2019

clickhouse-client -q "select arrayStringConcat(arrayMap(x->toString (cityHash64(x)) , range(1000)),' ') from numbers(100000);" >test.json
time cat test.json | curl "http://localhost:8123/?query=INSERT%20INTO%20log%20format%20CSV" --data-binary @-

clickhouse-client -q "DROP TABLE IF EXISTS log"
clickhouse-client -q "CREATE TABLE log (row String) ENGINE = MergeTree ORDER BY tuple()"

19.13.4.32
real	0m3.505s user	0m0.352s sys	0m1.836s
real	0m3.501s user	0m0.300s sys	0m1.912s

19.13.5.44
real	0m4.554s user	0m0.308s sys	0m1.924s
real	0m4.544s user	0m0.348s sys	0m1.888s

clickhouse-client -q "DROP TABLE IF EXISTS log"
clickhouse-client -q "CREATE TABLE log (row String) ENGINE = Log"

19.13.4.32
real	0m3.455s user	0m0.256s sys	0m2.008s
real	0m3.422s user	0m0.324s sys	0m1.924s

19.13.5.44
real	0m4.458s user	0m0.296s sys	0m1.968s
real	0m4.450s user	0m0.360s sys	0m1.916s

clickhouse-client -q "DROP TABLE IF EXISTS log"
clickhouse-client -q "CREATE TABLE log (row String) ENGINE = Memory"

19.13.4.32
real	0m3.007s user	0m0.364s sys	0m1.892s
real	0m3.017s user	0m0.236s sys	0m2.024s

19.13.5.44
real	0m4.036s user	0m0.356s sys	0m1.932s
real	0m4.049s user	0m0.364s sys	0m1.924s

time curl "http://localhost:8123/?query=INSERT%20INTO%20log%20format%20CSV" --data-binary @test.json

19.13.4.32
real	0m2.717s user	0m0.316s sys	0m1.156s
real	0m2.727s user	0m0.212s sys	0m1.264s

19.13.5.44
real	0m3.726s user	0m0.304s sys	0m1.188s
real	0m3.714s user	0m0.324s sys	0m1.152s

time clickhouse-client -q 'insert into log format CSV' <test.json

19.13.4.32
real	0m2.394s user	0m0.768s sys	0m0.668s
real	0m2.304s user	0m0.704s sys	0m0.696s

19.13.5.44
real	0m2.328s user	0m0.704s sys	0m0.684s
real	0m2.289s user	0m0.676s sys	0m0.704s

@den-crane
Copy link
Author

19.19.1.1957

time curl "http://localhost:8123/?query=INSERT%20INTO%20log%20format%20CSV" --data-binary @test.json

real 0m2.858s
user 0m0.240s
sys 0m1.111s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment