Skip to content

Instantly share code, notes, and snippets.

@TomAugspurger
Created May 20, 2025 21:55
Show Gist options
  • Save TomAugspurger/2235eb1d1a13f1e6158d4a24526f633e to your computer and use it in GitHub Desktop.
Save TomAugspurger/2235eb1d1a13f1e6158d4a24526f633e to your computer and use it in GitHub Desktop.
Ran query=1 in 12.8907s
Ran query=1 in 1.4123s
Ran query=1 in 2.0518s
Ran query=2 in 1.9878s
Ran query=2 in 1.1087s
Ran query=2 in 0.5492s
Ran query=3 in 6.8526s
Ran query=3 in 3.0100s
Ran query=3 in 3.1382s
Ran query=4 in 1.5490s
Ran query=4 in 1.5338s
Ran query=4 in 1.8743s
Ran query=5 in 3.4550s
Ran query=5 in 2.2236s
Ran query=5 in 2.4313s
Ran query=6 in 0.9503s
Ran query=6 in 0.8929s
Ran query=6 in 0.9205s
Ran query=7 in 3.4648s
Ran query=7 in 3.0649s
Ran query=7 in 2.8792s
Ran query=8 in 4.6277s
Ran query=8 in 3.4733s
Ran query=8 in 3.1357s
Ran query=9 in 4.7746s
Ran query=9 in 4.9289s
Ran query=9 in 5.0049s
Ran query=10 in 3.7263s
Ran query=10 in 3.1842s
Ran query=10 in 3.1969s
Ran query=11 in 0.7832s
Ran query=11 in 0.8234s
Ran query=11 in 0.6735s
Ran query=12 in 1.5903s
Ran query=12 in 1.6872s
Ran query=12 in 1.8504s
Ran query=13 in 4.4938s
Ran query=13 in 3.1299s
Ran query=13 in 2.2782s
Ran query=14 in 1.4744s
Ran query=14 in 1.2765s
Ran query=14 in 1.9716s
Ran query=15 in 1.4932s
Ran query=15 in 1.8029s
Ran query=15 in 1.1949s
Ran query=16 in 1.8971s
Ran query=16 in 1.8700s
Ran query=16 in 1.6559s
Ran query=17 in 2.6281s
Ran query=17 in 2.4437s
Ran query=17 in 3.0099s
Ran query=18 in 3.6904s
Ran query=18 in 5.8668s
Ran query=18 in 5.0826s
Ran query=19 in 1.8802s
Ran query=19 in 1.4015s
Ran query=19 in 1.7550s
Ran query=20 in 2.5248s
Ran query=20 in 2.3329s
Ran query=20 in 2.5443s
Ran query=21 in 4.1383s
Ran query=21 in 4.5063s
Ran query=21 in 4.0344s
Ran query=22 in 0.7127s
Ran query=22 in 0.7057s
Ran query=22 in 0.8337s
Iteration Summary
=======================================
query: 1
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.4123
max time : 12.8907
mean time: 5.4516
=======================================
query: 2
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 0.5492
max time : 1.9878
mean time: 1.2152
=======================================
query: 3
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.0100
max time : 6.8526
mean time: 4.3336
=======================================
query: 4
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.5338
max time : 1.8743
mean time: 1.6524
=======================================
query: 5
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 2.2236
max time : 3.4550
mean time: 2.7033
=======================================
query: 6
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 0.8929
max time : 0.9503
mean time: 0.9212
=======================================
query: 7
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 2.8792
max time : 3.4648
mean time: 3.1363
=======================================
query: 8
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.1357
max time : 4.6277
mean time: 3.7456
=======================================
query: 9
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 4.7746
max time : 5.0049
mean time: 4.9028
=======================================
query: 10
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.1842
max time : 3.7263
mean time: 3.3691
=======================================
query: 11
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 0.6735
max time : 0.8234
mean time: 0.7600
=======================================
query: 12
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.5903
max time : 1.8504
mean time: 1.7093
=======================================
query: 13
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 2.2782
max time : 4.4938
mean time: 3.3006
=======================================
query: 14
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.2765
max time : 1.9716
mean time: 1.5742
=======================================
query: 15
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.1949
max time : 1.8029
mean time: 1.4970
=======================================
query: 16
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.6559
max time : 1.8971
mean time: 1.8077
=======================================
query: 17
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 2.4437
max time : 3.0099
mean time: 2.6939
=======================================
query: 18
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.6904
max time : 5.8668
mean time: 4.8799
=======================================
query: 19
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.4015
max time : 1.8802
mean time: 1.6789
=======================================
query: 20
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 2.3329
max time : 2.5443
mean time: 2.4673
=======================================
query: 21
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 4.0344
max time : 4.5063
mean time: 4.2263
=======================================
query: 22
path: /data/cudf-sf-1000
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 0.7057
max time : 0.8337
mean time: 0.7507
=======================================
<Worker 'ucx://127.0.0.1:20361', name: 6, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 863.96 ms (avg 87.97 ns)
- event-loop-init-gpu-data-send: 1.89 s (avg 192.29 ns)
- event-loop-metadata-recv: 4.31 s (avg 438.79 ns)
- event-loop-metadata-send: 468.60 ms (avg 47.71 ns)
- event-loop-post-incoming-chunk-recv: 1.63 s (avg 166.08 ns)
- event-loop-total: 26.15 s (avg 2.22 us)
- shuffle-payload-recv: 460.21 GiB (avg 44.87 MiB)
- shuffle-payload-send: 479.58 GiB (avg 43.08 MiB)
- spill-bytes-device-to-host: 1.56 GiB (avg 99.82 MiB)
- spill-bytes-host-to-device: 9.76 GiB (avg 21.26 MiB)
- spill-bytes-recv-to-host: 2.36 GiB (avg 105.17 MiB)
- spill-time-device-to-host: 434.50 ms (avg 27.16 ms)
- spill-time-host-to-device: 1.00 s (avg 2.13 ms)
<Worker 'ucx://127.0.0.1:58413', name: 7, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 581.54 ms (avg 75.66 ns)
- event-loop-init-gpu-data-send: 950.71 ms (avg 123.68 ns)
- event-loop-metadata-recv: 5.68 s (avg 739.46 ns)
- event-loop-metadata-send: 509.41 ms (avg 66.27 ns)
- event-loop-post-incoming-chunk-recv: 1.35 s (avg 176.13 ns)
- event-loop-total: 24.48 s (avg 2.54 us)
- shuffle-payload-recv: 479.29 GiB (avg 46.07 MiB)
- shuffle-payload-send: 443.43 GiB (avg 45.44 MiB)
- spill-bytes-host-to-device: 0.00 B (avg 0.00 B)
- spill-time-host-to-device: 7.18 ms (avg 14.84 us)
<Worker 'ucx://127.0.0.1:60532', name: 3, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 766.70 ms (avg 40.86 ns)
- event-loop-init-gpu-data-send: 2.27 s (avg 121.17 ns)
- event-loop-metadata-recv: 4.66 s (avg 248.59 ns)
- event-loop-metadata-send: 857.56 ms (avg 45.70 ns)
- event-loop-post-incoming-chunk-recv: 2.67 s (avg 142.36 ns)
- event-loop-total: 36.44 s (avg 1.74 us)
- shuffle-payload-recv: 406.27 GiB (avg 41.88 MiB)
- shuffle-payload-send: 407.91 GiB (avg 38.32 MiB)
- spill-bytes-host-to-device: 0.00 B (avg 0.00 B)
- spill-time-host-to-device: 6.43 ms (avg 14.81 us)
<Worker 'ucx://127.0.0.1:36014', name: 4, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 834.30 ms (avg 44.51 ns)
- event-loop-init-gpu-data-send: 2.39 s (avg 127.47 ns)
- event-loop-metadata-recv: 5.71 s (avg 304.58 ns)
- event-loop-metadata-send: 1.06 s (avg 56.59 ns)
- event-loop-post-incoming-chunk-recv: 1.28 s (avg 68.05 ns)
- event-loop-total: 36.44 s (avg 1.74 us)
- shuffle-payload-recv: 447.98 GiB (avg 44.33 MiB)
- shuffle-payload-send: 480.68 GiB (avg 45.37 MiB)
- spill-bytes-host-to-device: 1.56 GiB (avg 3.48 MiB)
- spill-time-host-to-device: 169.44 ms (avg 369.14 us)
<Worker 'ucx://127.0.0.1:53414', name: 0, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 765.27 ms (avg 40.94 ns)
- event-loop-init-gpu-data-send: 1.92 s (avg 102.92 ns)
- event-loop-metadata-recv: 6.34 s (avg 338.90 ns)
- event-loop-metadata-send: 688.83 ms (avg 36.85 ns)
- event-loop-post-incoming-chunk-recv: 2.20 s (avg 117.94 ns)
- event-loop-total: 37.52 s (avg 1.80 us)
- shuffle-payload-recv: 419.09 GiB (avg 41.88 MiB)
- shuffle-payload-send: 453.42 GiB (avg 49.39 MiB)
- spill-bytes-host-to-device: 0.00 B (avg 0.00 B)
- spill-time-host-to-device: 6.70 ms (avg 14.96 us)
<Worker 'ucx://127.0.0.1:63846', name: 5, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 549.49 ms (avg 77.54 ns)
- event-loop-init-gpu-data-send: 717.65 ms (avg 101.27 ns)
- event-loop-metadata-recv: 4.34 s (avg 611.89 ns)
- event-loop-metadata-send: 519.23 ms (avg 73.27 ns)
- event-loop-post-incoming-chunk-recv: 2.39 s (avg 337.84 ns)
- event-loop-total: 22.93 s (avg 2.53 us)
- shuffle-payload-recv: 512.57 GiB (avg 47.28 MiB)
- shuffle-payload-send: 417.57 GiB (avg 41.56 MiB)
- spill-bytes-host-to-device: 330.92 MiB (avg 669.69 KiB)
- spill-bytes-recv-to-host: 25.30 MiB (avg 12.65 MiB)
- spill-time-host-to-device: 43.77 ms (avg 86.50 us)
<Worker 'ucx://127.0.0.1:18580', name: 2, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 764.07 ms (avg 171.63 ns)
- event-loop-init-gpu-data-send: 596.63 ms (avg 134.02 ns)
- event-loop-metadata-recv: 4.87 s (avg 1.09 us)
- event-loop-metadata-send: 238.26 ms (avg 53.52 ns)
- event-loop-post-incoming-chunk-recv: 1.65 s (avg 371.22 ns)
- event-loop-total: 20.15 s (avg 3.31 us)
- shuffle-payload-recv: 518.39 GiB (avg 47.56 MiB)
- shuffle-payload-send: 506.65 GiB (avg 49.56 MiB)
- spill-bytes-device-to-host: 2.93 GiB (avg 176.54 MiB)
- spill-bytes-host-to-device: 23.26 GiB (avg 46.16 MiB)
- spill-bytes-recv-to-host: 1.96 GiB (avg 87.20 MiB)
- spill-time-device-to-host: 742.96 ms (avg 43.70 ms)
- spill-time-host-to-device: 2.24 s (avg 4.34 ms)
<Worker 'ucx://127.0.0.1:56951', name: 1, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 740.26 ms (avg 39.54 ns)
- event-loop-init-gpu-data-send: 1.83 s (avg 97.93 ns)
- event-loop-metadata-recv: 6.14 s (avg 327.83 ns)
- event-loop-metadata-send: 712.03 ms (avg 38.04 ns)
- event-loop-post-incoming-chunk-recv: 1.87 s (avg 99.75 ns)
- event-loop-total: 36.47 s (avg 1.74 us)
- shuffle-payload-recv: 376.92 GiB (avg 41.49 MiB)
- shuffle-payload-send: 431.49 GiB (avg 44.40 MiB)
- spill-bytes-host-to-device: 0.00 B (avg 0.00 B)
- spill-time-host-to-device: 6.02 ms (avg 14.75 us)
Ran query=1 in 7.9231s
Ran query=1 in 1.6724s
Ran query=1 in 1.5966s
Ran query=2 in 4.2359s
Ran query=2 in 1.0091s
Ran query=2 in 1.1056s
Ran query=3 in 7.5936s
Ran query=3 in 4.7181s
Ran query=3 in 2.4843s
Ran query=4 in 2.1369s
Ran query=4 in 1.6183s
Ran query=4 in 1.5935s
Ran query=5 in 4.8633s
Ran query=5 in 2.7236s
Ran query=5 in 3.0714s
Ran query=6 in 1.0174s
Ran query=6 in 0.9717s
Ran query=6 in 0.9892s
Ran query=7 in 3.9093s
Ran query=7 in 3.7906s
Ran query=7 in 3.7526s
Ran query=8 in 7.2237s
Ran query=8 in 5.5298s
Ran query=8 in 6.3416s
Ran query=9 in 7.4402s
Ran query=9 in 7.1270s
Ran query=9 in 7.8724s
Ran query=10 in 4.2458s
Ran query=10 in 3.4588s
Ran query=10 in 3.5693s
Ran query=11 in 0.8489s
Ran query=11 in 0.7559s
Ran query=11 in 0.7712s
Ran query=12 in 1.9073s
Ran query=12 in 1.7097s
Ran query=12 in 1.7518s
Ran query=13 in 6.3178s
Ran query=13 in 3.4989s
Ran query=13 in 3.0033s
Ran query=14 in 2.2338s
Ran query=14 in 1.7875s
Ran query=14 in 2.1870s
Ran query=15 in 2.2244s
Ran query=15 in 2.9984s
Ran query=15 in 1.7473s
Ran query=16 in 1.4402s
Ran query=16 in 1.4997s
Ran query=16 in 2.0001s
Ran query=17 in 3.0781s
Ran query=17 in 3.2931s
Ran query=17 in 3.2081s
Ran query=18 in 3.8204s
Ran query=18 in 4.2376s
Ran query=18 in 3.6765s
Ran query=19 in 2.2028s
Ran query=19 in 2.3775s
Ran query=19 in 1.9741s
Ran query=20 in 3.6458s
Ran query=20 in 2.8306s
Ran query=20 in 3.0041s
Ran query=21 in 5.7110s
Ran query=21 in 5.4988s
Ran query=21 in 5.5030s
Ran query=22 in 1.4234s
Ran query=22 in 1.5049s
Ran query=22 in 1.4893s
Iteration Summary
=======================================
query: 1
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.5966
max time : 7.9231
mean time: 3.7307
=======================================
query: 2
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.0091
max time : 4.2359
mean time: 2.1169
=======================================
query: 3
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 2.4843
max time : 7.5936
mean time: 4.9320
=======================================
query: 4
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.5935
max time : 2.1369
mean time: 1.7829
=======================================
query: 5
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 2.7236
max time : 4.8633
mean time: 3.5528
=======================================
query: 6
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 0.9717
max time : 1.0174
mean time: 0.9928
=======================================
query: 7
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.7526
max time : 3.9093
mean time: 3.8175
=======================================
query: 8
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 5.5298
max time : 7.2237
mean time: 6.3650
=======================================
query: 9
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 7.1270
max time : 7.8724
mean time: 7.4798
=======================================
query: 10
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.4588
max time : 4.2458
mean time: 3.7580
=======================================
query: 11
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 0.7559
max time : 0.8489
mean time: 0.7920
=======================================
query: 12
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.7097
max time : 1.9073
mean time: 1.7896
=======================================
query: 13
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.0033
max time : 6.3178
mean time: 4.2733
=======================================
query: 14
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.7875
max time : 2.2338
mean time: 2.0694
=======================================
query: 15
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.7473
max time : 2.9984
mean time: 2.3234
=======================================
query: 16
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.4402
max time : 2.0001
mean time: 1.6467
=======================================
query: 17
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.0781
max time : 3.2931
mean time: 3.1931
=======================================
query: 18
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 3.6765
max time : 4.2376
mean time: 3.9115
=======================================
query: 19
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.9741
max time : 2.3775
mean time: 2.1848
=======================================
query: 20
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 2.8306
max time : 3.6458
mean time: 3.1601
=======================================
query: 21
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 5.4988
max time : 5.7110
mean time: 5.5710
=======================================
query: 22
path: /data-rs/scale-1000-1kschema
executor: streaming
scheduler: distributed
blocksize: None
shuffle_method: rapidsmpf
broadcast_join_limit: None
n_workers: 8
threads: 1
spill_device: 0.5
rapidsmpf_spill: False
iterations: 3
---------------------------------------
min time : 1.4234
max time : 1.5049
mean time: 1.4725
=======================================
<Worker 'ucx://127.0.0.1:51322', name: 6, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 1.04 s (avg 74.59 ns)
- event-loop-init-gpu-data-send: 1.80 s (avg 128.90 ns)
- event-loop-metadata-recv: 6.47 s (avg 463.04 ns)
- event-loop-metadata-send: 800.46 ms (avg 57.32 ns)
- event-loop-post-incoming-chunk-recv: 2.85 s (avg 204.11 ns)
- event-loop-total: 36.53 s (avg 2.18 us)
- shuffle-payload-recv: 578.72 GiB (avg 22.80 MiB)
- shuffle-payload-send: 550.84 GiB (avg 20.88 MiB)
- spill-bytes-host-to-device: 6.16 GiB (avg 8.38 MiB)
- spill-bytes-recv-to-host: 1.53 GiB (avg 27.48 MiB)
- spill-time-host-to-device: 782.98 ms (avg 1.04 ms)
<Worker 'ucx://127.0.0.1:13431', name: 3, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 1.24 s (avg 205.49 ns)
- event-loop-init-gpu-data-send: 2.26 s (avg 373.55 ns)
- event-loop-metadata-recv: 4.94 s (avg 816.25 ns)
- event-loop-metadata-send: 358.08 ms (avg 59.11 ns)
- event-loop-post-incoming-chunk-recv: 4.29 s (avg 707.81 ns)
- event-loop-total: 31.21 s (avg 3.83 us)
- shuffle-payload-recv: 645.05 GiB (avg 23.59 MiB)
- shuffle-payload-send: 610.82 GiB (avg 23.70 MiB)
- spill-bytes-device-to-host: 3.11 GiB (avg 27.47 MiB)
- spill-bytes-host-to-device: 18.87 GiB (avg 23.57 MiB)
- spill-bytes-recv-to-host: 7.54 GiB (avg 28.80 MiB)
- spill-time-device-to-host: 985.47 ms (avg 8.50 ms)
- spill-time-host-to-device: 2.10 s (avg 2.56 ms)
<Worker 'ucx://127.0.0.1:17919', name: 7, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 1.19 s (avg 130.72 ns)
- event-loop-init-gpu-data-send: 1.64 s (avg 179.53 ns)
- event-loop-metadata-recv: 5.20 s (avg 570.83 ns)
- event-loop-metadata-send: 646.73 ms (avg 70.97 ns)
- event-loop-post-incoming-chunk-recv: 5.61 s (avg 615.61 ns)
- event-loop-total: 36.06 s (avg 3.12 us)
- shuffle-payload-recv: 640.52 GiB (avg 23.43 MiB)
- shuffle-payload-send: 601.35 GiB (avg 21.63 MiB)
- spill-bytes-device-to-host: 3.97 GiB (avg 27.46 MiB)
- spill-bytes-host-to-device: 22.67 GiB (avg 28.52 MiB)
- spill-bytes-recv-to-host: 10.58 GiB (avg 28.90 MiB)
- spill-time-device-to-host: 1.23 s (avg 8.34 ms)
- spill-time-host-to-device: 2.50 s (avg 3.07 ms)
<Worker 'ucx://127.0.0.1:21263', name: 5, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 1.18 s (avg 70.69 ns)
- event-loop-init-gpu-data-send: 2.02 s (avg 120.98 ns)
- event-loop-metadata-recv: 5.73 s (avg 342.53 ns)
- event-loop-metadata-send: 1.08 s (avg 64.51 ns)
- event-loop-post-incoming-chunk-recv: 2.64 s (avg 157.82 ns)
- event-loop-total: 37.77 s (avg 1.93 us)
- shuffle-payload-recv: 547.04 GiB (avg 21.95 MiB)
- shuffle-payload-send: 582.51 GiB (avg 22.49 MiB)
- spill-bytes-device-to-host: 137.40 MiB (avg 27.48 MiB)
- spill-bytes-host-to-device: 7.05 GiB (avg 9.86 MiB)
- spill-bytes-recv-to-host: 1.65 GiB (avg 27.64 MiB)
- spill-time-device-to-host: 43.23 ms (avg 8.65 ms)
- spill-time-host-to-device: 915.09 ms (avg 1.25 ms)
<Worker 'ucx://127.0.0.1:34281', name: 0, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 1.10 s (avg 65.68 ns)
- event-loop-init-gpu-data-send: 2.67 s (avg 160.11 ns)
- event-loop-metadata-recv: 5.27 s (avg 315.89 ns)
- event-loop-metadata-send: 1.03 s (avg 61.94 ns)
- event-loop-post-incoming-chunk-recv: 2.71 s (avg 162.62 ns)
- event-loop-total: 38.10 s (avg 1.96 us)
- shuffle-payload-recv: 541.70 GiB (avg 21.95 MiB)
- shuffle-payload-send: 571.18 GiB (avg 23.25 MiB)
- spill-bytes-host-to-device: 3.86 GiB (avg 5.46 MiB)
- spill-bytes-recv-to-host: 494.42 MiB (avg 27.47 MiB)
- spill-time-host-to-device: 481.69 ms (avg 665.32 us)
<Worker 'ucx://127.0.0.1:46657', name: 4, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 1.14 s (avg 40.03 ns)
- event-loop-init-gpu-data-send: 3.86 s (avg 135.71 ns)
- event-loop-metadata-recv: 9.18 s (avg 323.10 ns)
- event-loop-metadata-send: 1.09 s (avg 38.26 ns)
- event-loop-post-incoming-chunk-recv: 2.15 s (avg 75.78 ns)
- event-loop-total: 55.17 s (avg 1.77 us)
- shuffle-payload-recv: 526.21 GiB (avg 21.55 MiB)
- shuffle-payload-send: 571.69 GiB (avg 21.96 MiB)
- spill-bytes-host-to-device: 1.68 GiB (avg 2.44 MiB)
- spill-bytes-recv-to-host: 137.25 MiB (avg 27.45 MiB)
- spill-time-host-to-device: 260.53 ms (avg 369.03 us)
<Worker 'ucx://127.0.0.1:16977', name: 1, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 1.18 s (avg 85.69 ns)
- event-loop-init-gpu-data-send: 2.33 s (avg 168.87 ns)
- event-loop-metadata-recv: 6.16 s (avg 447.08 ns)
- event-loop-metadata-send: 603.53 ms (avg 43.83 ns)
- event-loop-post-incoming-chunk-recv: 4.66 s (avg 338.52 ns)
- event-loop-total: 40.52 s (avg 2.45 us)
- shuffle-payload-recv: 592.12 GiB (avg 22.41 MiB)
- shuffle-payload-send: 611.74 GiB (avg 24.38 MiB)
- spill-bytes-device-to-host: 1.91 GiB (avg 30.57 MiB)
- spill-bytes-host-to-device: 17.42 GiB (avg 23.04 MiB)
- spill-bytes-recv-to-host: 7.01 GiB (avg 33.68 MiB)
- spill-time-device-to-host: 615.58 ms (avg 9.62 ms)
- spill-time-host-to-device: 1.89 s (avg 2.44 ms)
<Worker 'ucx://127.0.0.1:27414', name: 2, status: running, stored: 1, running: 0/1, ready: 0, comm: 0, waiting: 0> Statistics:
- event-loop-check-future-finish: 1.17 s (avg 84.63 ns)
- event-loop-init-gpu-data-send: 2.56 s (avg 184.15 ns)
- event-loop-metadata-recv: 6.43 s (avg 463.59 ns)
- event-loop-metadata-send: 606.66 ms (avg 43.71 ns)
- event-loop-post-incoming-chunk-recv: 3.08 s (avg 221.78 ns)
- event-loop-total: 38.37 s (avg 2.30 us)
- shuffle-payload-recv: 601.72 GiB (avg 22.69 MiB)
- shuffle-payload-send: 572.96 GiB (avg 22.49 MiB)
- spill-bytes-device-to-host: 1.10 GiB (avg 27.47 MiB)
- spill-bytes-host-to-device: 9.47 GiB (avg 12.44 MiB)
- spill-bytes-recv-to-host: 3.29 GiB (avg 29.00 MiB)
- spill-time-device-to-host: 332.36 ms (avg 8.11 ms)
- spill-time-host-to-device: 1.09 s (avg 1.40 ms)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment