Richard (Rick) Zamora rjzamora
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import os | |
| import shutil | |
| import time | |
| import numpy as np | |
| import dask.dataframe as dd | |
| from dask.dataframe.io.demo import names as name_list | |
| import fastavro as fa | |
| import cudf |
rjzamora
/ read_file_list.ipynb
Created
October 30, 2020 20:02
Simple example of mapping multiple parquet files to each dask_cudf.DataFrame partition.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
rjzamora
/ nvtabular_client_example.py
Created
December 10, 2020 22:58
NVTabular + Distributed Client Example
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from dask.distributed import Client | |
| cluster = "tcp://MachineA:8786" | |
| client = Client(cluster) | |
| … | |
| import nvtabular as nvt | |
| Workflow = nvt.Workflow(client=client, …) |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
rjzamora
/ dask_summit_scheduler_problems.ipynb
Last active
May 12, 2021 14:42
Simple examples of the Dataframe-based workflows targeted by the ongoing Scheduler-optimization project.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
rjzamora
/ dask_array_dataloader.ipynb
Created
November 24, 2021 21:07
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
rjzamora
/ remote_parquet_benchmark.py
Created
December 10, 2021 21:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import importlib | |
| import time | |
| import dask.dataframe as dd | |
| from dask.distributed import LocalCluster, Client | |
| try: | |
| from dask_cuda import LocalCUDACluster | |
| except ImportError: | |
| dask_cuda = None | |
| try: |
rjzamora
/ regenerate_layers.ipynb
Last active
February 16, 2022 16:30
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.