Skip to content

Instantly share code, notes, and snippets.

@ctlllll
ctlllll / longest_chinese_tokens_gpt4o.py
Created May 13, 2024 19:53
Longest Chinese tokens in gpt4o
import tiktoken
import langdetect
T = tiktoken.get_encoding("o200k_base")
length_dict = {}
for i in range(T.n_vocab):
try:
length_dict[i] = len(T.decode([i]))
except:
@mikeckennedy
mikeckennedy / syncify.py
Last active April 2, 2023 07:02
Convert an async method to a synchronous one.
import asyncio
import functools
from typing import Any, Coroutine
loop = asyncio.new_event_loop()
def run(async_coroutine: Coroutine[Any, Any, Any]):
"""
Convert an async method to a synchronous one.
@brews
brews / papermill-test-workflow.yaml
Last active February 12, 2022 00:38
Simple Jupyter notebook with a "parameter" tagged cell for running with papermill. Includes Argo Workflow running notebook on jupyterhub container image.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: papermill-test-
spec:
entrypoint: main
templates:
- name: main
inputs:
@brews
brews / dask-internal-process-demo.yaml
Last active January 20, 2022 01:58
Example Argo Workflow creating a process-based dask.distributed job with a LocalCluster, internal to the pod's node.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: dask-internal-process-demo-
spec:
entrypoint: dask
activeDeadlineSeconds: 1800 # Safety first, kids!
templates:
- name: dask
script:
@brews
brews / make_dtr.py
Last active December 23, 2021 20:32
Downscale CMIP6 (dc6) sandbox scratch script to get tasmin, tasmax parameter files for each GCM and combine to create DTR parameter file.
# Get tasmin, tasmax parameter files for each GCM and combine to create DTR parameter file.
# This is all rough and likely to break. Sorry.
import logging
import pathlib
import dearprudence
import dearprudence.validation
PARAMS_DIR = pathlib.Path("/Users/bmalevich/Projects/downscaleCMIP6/workflows/parameters")
@brews
brews / add_cyclic.py
Created October 26, 2021 17:30
xarray-native approach to adding a cyclic pixel (aka wrap-around pixel) to arrays in an xarray.Dataset.
import xarray as xr
def add_cyclic(ds, dim):
"""
Adds wrap-around, appending first value to end of data for named dimension.
Basically an xarray version of ``cartopy.util.add_cyclic_point()``.
"""
return ds.map(
@brews
brews / xarray_wrap_around_pixel_demo.py
Last active October 7, 2024 14:40
Demo using cartopy.util.add_cyclic_point with an xarray Dataset to add a cyclic or wrap-around pixel to the `lon` dimension. This can be useful for plotting with `cartopy` or regridding with `xesmf`.
"""
Demo using cartopy.util.add_cyclic_point with an xarray Dataset to
add a cyclic or wrap-around pixel to the `lon` dimension. This can be useful
for plotting with `cartopy` or regridding with `xesmf`.
"""
import xarray as xr
from cartopy.util import add_cyclic_point
@jph00
jph00 / py.md
Last active May 31, 2022 06:16
Organized and hyperlinked index to every module, function, and class in the Python standard library

All of the python 3.9 standard library

For a version without the collapsible details sections (so you can search the whole thing in your browser), click here.

@nicoguaro
nicoguaro / neon_example.png
Last active September 28, 2024 18:55
Plots with neon-glow in Matplotlib.
neon_example.png
@brews
brews / dask-test-workflow.yaml
Last active October 31, 2023 16:22
Argo Workflow demo to launch kubernetes dask distributed job
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: dask-test-
spec:
entrypoint: dask
activeDeadlineSeconds: 1800 # Safety first, kids!
templates:
- name: dask
script: