Tom Augspurger TomAugspurger
TomAugspurger
/ file-handler.diff
Created
March 20, 2026 12:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| diff --git a/python/cudf_polars/cudf_polars/experimental/benchmarks/utils.py b/python/cudf_polars/cudf_polars/experimental/benchmarks/utils.py | |
| index f40889b68c..375e1dbebb 100644 | |
| --- a/python/cudf_polars/cudf_polars/experimental/benchmarks/utils.py | |
| +++ b/python/cudf_polars/cudf_polars/experimental/benchmarks/utils.py | |
| @@ -1569,9 +1569,9 @@ def run_polars_query( | |
| for i in range(args.iterations): | |
| if _HAS_STRUCTLOG and run_config.collect_traces: | |
| - setup_logging(q_id, i) | |
| + setup_logging(q_id, i, run_config.run_id) |
TomAugspurger
/ pdsh_results.jsonl
Created
March 19, 2026 13:06
This file has been truncated, but you can view the full file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"queries": [9], "suffix": "", "executor": "streaming", "runtime": "rapidsmpf", "stream_policy": null, "cluster": "single", "scheduler": "synchronous", "n_workers": 1, "versions": {"cudf_polars": {"version": "26.06.00", "commit": ""}, "polars": "1.37.1", "python": "3.13.3", "rapidsmpf": {"version": "26.06.00a4", "commit": "4f26308f0fadcde236583b2cfda9a059b9c0c878"}}, "records": {"9": [{"query": 9, "iteration": 0, "duration": 6.197310272604227, "shuffle_stats": null, "traces": [{"scope": "plan", "plan": {"roots": ["261005036"], "nodes": {"261005036": {"id": "261005036", "children": ["3966771035"], "schema": {"nation": "STRING", "o_year": "INT32", "sum_profit": "DECIMAL128"}, "properties": {"by": ["nation", "o_year"], "order": ["ASCENDING", "DESCENDING"]}, "type": "Sort"}, "3966771035": {"id": "3966771035", "children": ["2303697351"], "schema": {"nation": "STRING", "o_year": "INT32", "sum_profit": "DECIMAL128"}, "properties": {}, "type": "Repartition"}, "2303697351": {"id": "2303697351", "children": ["245703849 |
TomAugspurger
/ slow_spill.py
Created
November 14, 2025 20:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import rapidsmpf | |
| import pylibcudf as plc | |
| import pyarrow as pa | |
| import numpy as np | |
| import rmm.mr | |
| import rmm.pylibrmm.stream | |
| import rapidsmpf.communicator.single | |
| import rapidsmpf.shuffler | |
| import rapidsmpf.buffer.resource | |
| import rapidsmpf.buffer.buffer |
TomAugspurger
/ q4-spilling.ipynb
Created
November 13, 2025 16:06
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
TomAugspurger
/ packages.diff
Created
November 10, 2025 17:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- ok.txt 2025-11-10 09:38:59.410995539 -0800 | |
| +++ error.txt 2025-11-10 09:39:15.465981068 -0800 | |
| @@ -20,7 +20,7 @@ | |
| aws-c-sdkutils 0.2.4 h7e655bb_2 conda-forge | |
| aws-checksums 0.2.7 h7e655bb_3 conda-forge | |
| aws-crt-cpp 0.35.0 h719b17a_2 conda-forge | |
| -aws-sdk-cpp 1.11.606 h522d481_5 conda-forge | |
| +aws-sdk-cpp 1.11.606 h522d481_6 conda-forge | |
| aws-xray-sdk 2.15.0 pyhd8ed1ab_0 conda-forge | |
| azure-core-cpp 1.16.1 h3a458e0_0 conda-forge |
TomAugspurger
/ log.txt
Created
November 7, 2025 21:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (cudf-polars-bench) root@gpu-h100-0161:/app# for query in {1..22}; do | |
| > nsys profile \ | |
| > -o "/data/profiles/rapidsmpf.q$query.1k" -f true \ | |
| > --trace=nvtx,cuda \ | |
| > --nvtx-domain-exclude=CCCL,rapidsmpf,libkvikio \ | |
| > --cuda-memory-usage=true \ | |
| > python -m cudf_polars.experimental.benchmarks.pdsh \ | |
| > --executor="streaming" \ | |
| > --runtime="rapidsmpf" \ | |
| > --path="/data/tpch-rs/scale-1000" \ |
TomAugspurger
/ page.html
Created
November 7, 2025 20:05
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <title>Benchmark Results Explorer</title> | |
| <link href="https://unpkg.com/[email protected]/dist/css/tabulator.min.css" rel="stylesheet"> | |
| <style> | |
| body { | |
| font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif; |
TomAugspurger
/ log.txt
Created
November 7, 2025 17:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| root@gpu-h100-0261:/app# export TERM=linux | |
| root@gpu-h100-0261:/app# unset UCX_NET_DEVICES | |
| root@gpu-h100-0261:/app# export UCX_PROTO_ENABLE="y" | |
| root@gpu-h100-0261:/app# export UCX_RNDV_PIPELINE_ERROR_HANDLING="y" | |
| root@gpu-h100-0261:/app# export UCX_MAX_RNDV_RAILS="1" | |
| root@gpu-h100-0261:/app# export DASK_DISTRIBUTED__COMM__TIMEOUTS__CONNECT="300" | |
| root@gpu-h100-0261:/app# export DASK_DISTRIBUTED__COMM__UCX__CONNECT_TIMEOUT="300" | |
| root@gpu-h100-0261:/app# | |
| root@gpu-h100-0261:/app# . /app/.venv/bin/activate | |
| (cudf-polars-bench) root@gpu-h100-0261:/app# |
TomAugspurger
/ 2025-11-07.0.txt
Created
November 7, 2025 13:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (cudf-polars-bench) root@gpu-h100-0468:/app# for query in {1..22}; do | |
| > python -m cudf_polars.experimental.benchmarks.pdsh \ | |
| > --executor="streaming" \ | |
| > --runtime="rapidsmpf" \ | |
| > --path="/data/tpch-rs/scale-1000" \ | |
| > --suffix="" \ | |
| > --stream-policy="pool" \ | |
| > --n-workers 1 \ | |
| > --no-print-results \ | |
| > --no-summarize \ |
TomAugspurger
/ debug.cpp
Created
October 27, 2025 17:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // ----------------------------------------------------------------------------- | |
| // Reproducer (translated from Python) | |
| // ----------------------------------------------------------------------------- | |
| #include <cudf/column/column.hpp> | |
| #include <cudf/column/column_factories.hpp> | |
| #include <cudf/strings/contains.hpp> | |
| #include <cudf/strings/regex/regex_program.hpp> | |
| #include <cudf/strings/strings_column_view.hpp> | |
| #include <cudf/types.hpp> |
NewerOlder