Skip to content

Instantly share code, notes, and snippets.

View odashi's full-sized avatar
🏠
Working from home

Yusuke Oda odashi

🏠
Working from home
View GitHub Profile
@odashi
odashi / parallel.py
Created September 10, 2024 19:24
Parallel command invocation with specified workers
import argparse as ap
import dataclasses as dc
import multiprocessing as mp
import subprocess as sp
import sys
def parse_args() -> ap.Namespace:
p = ap.ArgumentParser("Runs parallel jobs")
@odashi
odashi / install_local_python.sh
Last active September 1, 2024 18:02
Local Python installer
#!/bin/bash
# This script installs a Python environment built from source.
# Usage:
# URL='https://gist.githubusercontent.com/odashi/...copy-paste raw path.../install_local_python.sh'
# bash <(curl -sL ${URL}) ./env 3.10.14 64
set -eu -o pipefail
if [ $# -ne 3 ]; then
>&2 echo "Usage: install_local_python.sh INSTALL_DIR MAJOR.MINOR.PATCH BUILD_THREADS"
@odashi
odashi / download_c4_en.bash
Last active January 14, 2024 00:22
Downloads AllenAI mC4 dataset
#!/bin/bash
url_prefix=https://huggingface.co/datasets/allenai/c4/resolve/1ddc917116b730e1859edef32896ec5c16be51d0/en
function download() {
dataset_name=$1; shift
splits=$1; shift
last_split=$(perl -e "print ${splits} - 1")
mkdir -p ${dataset_name}
for i in $(seq 0 ${last_split}); do
@odashi
odashi / gemini_adc.py
Last active December 20, 2023 05:59
Example to call Gemini API with ADC
# You may need to install the following libraries:
# * google-auth
# * requests
import json
from typing import Any
import google.auth
import google.auth.credentials
import google.auth.transport.requests
@odashi
odashi / auth.tsx
Created December 9, 2023 14:48
Firebase Authentication provider with next.js
"use client";
import { getApp, getApps, initializeApp } from "firebase/app";
import {
getAuth,
GoogleAuthProvider,
signInWithPopup,
User,
} from "firebase/auth";
import {
@odashi
odashi / BUILD
Last active November 17, 2022 06:27
Bazel rule to generate Python structs from OpenAPI documents
load("@pip_deps//:requirements.bzl", "requirement")
py_binary(
name = "datamodel_codegen",
srcs = ["datamodel_codegen_main.py"],
main = "datamodel_codegen_main.py",
visibility = ["//visibility:public"],
deps = [
requirement("datamodel-code-generator"),
],
@odashi
odashi / class_fastapi_app.py
Last active April 13, 2023 15:14
FastAPI server defined on a class
# Sometimes you want to define your FastAPI app within a class for some reason, e.g.,
# you wanted to initialize the servers at the point you intended, not the point of import.
#
# In this case, @app.method decorator doesn't work as-is with instance methods due to the
# difference of its signature.
#
# You need to manually call the decorator as a usual function after obtaining self (e.g.,
# in the __init__ method like below) rather than using the decorator syntax.
class MyApp:
@odashi
odashi / nai_prompt_generator.py
Last active October 26, 2022 14:57
Colab UI to generate NovelAI prompts.
!pip install ipywidgets &> /dev/null
import ipywidgets as wgt
controls = wgt.VBox([])
add_button = wgt.Button(description="Add", icon="plus")
pos_prompt_area = wgt.Textarea(placeholder="Positive prompts appear here.")
neg_prompt_area = wgt.Textarea(placeholder="Negative prompts appear here.")
ui = wgt.VBox([pos_prompt_area, neg_prompt_area, add_button, controls])
def generate_prompt(change):
@odashi
odashi / should_mute_on_twitter.txt
Last active December 26, 2022 13:02
should_mute.txt
あまり大きな声で言えないのですが
あまり知られてないのですが
ありえない
あんまり言いたくないけど
いい加減にしてほしい
いい歳してお恥ずかしいのですが
悪用厳禁なのですが
安心してください
言いたくない過去を言うと
言わせてください
@odashi
odashi / delete_old_tweets.py
Last active July 26, 2023 20:35
Cloud Functions script to delete old tweets.
import datetime
import logging
import os
from google.cloud import logging as cloud_logging
import tweepy
THRESHOLD_DAYS = 3
BATCH_SIZE = 25