Skip to content

Instantly share code, notes, and snippets.

@JimmyWhitaker
JimmyWhitaker / gpu-type-pachyderm.json
Created October 8, 2020 16:46
Setting the GPU type in a Pachyderm Pipeline
"scheduling_spec": {
"node_selector": {
"cloud.google.com/gke-accelerator": "nvidia-tesla-t4"
}
}
@JimmyWhitaker
JimmyWhitaker / pachyderm-resource-requests.json
Created October 8, 2020 21:17
Setting resource limits in a Pachyderm spec
"resource_limits": {
"gpu": {
"type": "nvidia.com/gpu",
"number": 1
}
},
"resource_requests": {
"memory": "2G",
"cpu": 1
},
{
"pipeline": {
"name": "inspect-input"
},
"description": "Inspect /pfs/ input.",
"transform": {
"cmd": [
"/bin/sh"
],
"stdin": [
ENDPOINT_URL=http://192.168.64.8:30600
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test
@JimmyWhitaker
JimmyWhitaker / image_gallary.py
Last active August 10, 2022 21:17
Streamlit Application for an image gallery, given a directory of images
import streamlit as st
import itertools
from pathlib import Path
import argparse
parser = argparse.ArgumentParser(description="Serve image from PFS")
parser.add_argument("--image-path", type=str, help="")
def paginator(label, items, items_per_page=10, on_sidebar=True):
"""Lets the user paginate a set of items.
@JimmyWhitaker
JimmyWhitaker / Bootstrap Labels with GPT-4.ipynb
Last active July 26, 2024 22:08
Bootstrap Labels with GPT-4
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.