Skip to content

Instantly share code, notes, and snippets.

View pentschev's full-sized avatar

Peter Andreas Entschev pentschev

52 followers · 0 following
  • NVIDIA
View GitHub Profile
@pentschev
pentschev / pynvml_query_memory.py
Last active April 28, 2020 08:22
Query used GPU memory with pynvml
import datetime
import getopt
import os
import sys
import time
import pynvml
def get_printable_util_mem(dev_count, peak_mem):
@pentschev
pentschev / occupancy_hang.txt
Created September 17, 2019 18:30
Thread 9 (Thread 0x7f2da4fe8700 (LWP 39329)):
Undefined command: "btr". Try "help".
Thread 9 (Thread 0x7f2da4fe8700 (LWP 39329)):
#0 0x00007f2dccf7b709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x000055787b3902d8 in PyCOND_TIMEDWAIT (cond=0x55787b5c2a38 <_PyRuntime+1208>, mut=0x55787b5c2a68 <_PyRuntime+1256>,
us=5000) at /home/conda/feedstock_root/build_artifacts/python_1562015400360/work/Python/condvar.h:90
#2 take_gil (tstate=0x55787f257a40)
at /home/conda/feedstock_root/build_artifacts/python_1562015400360/work/Python/ceval_gil.h:208
@pentschev
pentschev / dask_cudf_cuml.ipynb
Created September 3, 2019 19:55
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@pentschev
pentschev / ucx_setup.sh
Last active August 15, 2019 13:25
Script for setup of a UCX-enabled conda environment
#!/bin/bash
SOURCE_PATH=${HOME}/ucx-src
CONDA_ENV_NAME=ucx
RAPIDS_VERSION="0.9"
CUDA_VERSION="9.2"
# CUDA_VERSION="10.0"
if [ ! -d ${SOURCE_PATH} ]; then
mkdir ${SOURCE_PATH}
fi
@pentschev
pentschev / map_overlap_simple_10k_tcp.html
Created August 12, 2019 22:50
Task Streams for map_overlap(lambda x: x) with UCX and TCP
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@pentschev
pentschev / map_overlap_10k_tcp.html
Created August 12, 2019 22:10
Task Streams for map_overlap with UCX and TCP
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@pentschev
pentschev / map_overlap_10k_tcp.html
Created August 12, 2019 22:10
Task Streams for map_overlap with UCX and TCP
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@pentschev
pentschev / ucx_map_overlap_smooth.py
Last active August 16, 2019 10:06
UCX map overlap sample
import asyncio
import time
import numpy as np
import cupy
import numba
import dask.array as da
from dask_cuda import DGX, LocalCUDACluster
from dask.distributed import Client, wait
@pentschev
pentschev / dask_cupy_custom_kernel_output.py
Created August 9, 2019 20:27
Blog Post - Parallelizing Custom CuPy Kernels with Dask - Output
res.compute()
array([[0.000000e+00, 2.000000e+00, 4.000000e+00, ..., 2.042000e+03,
2.044000e+03, 2.046000e+03],
[1.024000e+03, 1.026000e+03, 1.028000e+03, ..., 3.066000e+03,
3.068000e+03, 3.070000e+03],
[2.048000e+03, 2.050000e+03, 2.052000e+03, ..., 4.090000e+03,
4.092000e+03, 4.094000e+03],
...,
[4.191232e+06, 4.191234e+06, 4.191236e+06, ..., 4.193274e+06,
4.193276e+06, 4.193278e+06],
@pentschev
pentschev / map_blocks_dispatch_add_broadcast.py
Created August 9, 2019 20:26
Blog Post - Parallelizing Custom CuPy Kernels with Dask - Map Blocks
res = da.map_blocks(dispatch_add_broadcast, dx, dy, dtype=cupy.float32)