Skip to content

Instantly share code, notes, and snippets.

View goddoe's full-sized avatar

Sungju Kim goddoe

View GitHub Profile
@goddoe
goddoe / reservoir_sampling.py
Created July 13, 2023 09:09
파일이 너무 커서 한 번에 메모리에 로드할 수 없는 경우, 간단한 무작위 샘플링은 불가능합니다. 그러나, 이 문제를 해결하기 위해 Reservoir Sampling이라는 알고리즘이 있습니다. 이 알고리즘은 스트림에서 무작위로 샘플을 선택하는 데 사용됩니다.
import random
def reservoir_sampling(file_name, k):
sample = []
with open(file_name, 'r') as f:
f.seek(0, 2) # 파일의 끝으로 이동
filesize = f.tell() # 파일의 크기를 얻음 (바이트)
random_set = sorted(random.sample(range(filesize), k))
@goddoe
goddoe / run_after_pid_done.sh
Created May 16, 2023 14:58
run after specific process done.
#!/usr/bin/env bash
PID=<SOME_PID>
while true; do
if ! ps -p PID> > /dev/null; then
./run.sh
break
fi
sleep 10
import logging
import sys
# Setup logging
logging.basicConfig(
format="%(asctime)s - %(levelname)s - %(name)s - %(message)s",
datefmt="%m/%d/%Y %H:%M:%S",
handlers=[logging.StreamHandler(sys.stdout)],
)
@goddoe
goddoe / text_gradient.html
Created February 20, 2023 17:24
text_gradient.html
<!-- Ref: https://daily-dev-tips.com/posts/making-gradient-text-with-tailwind-css/ -->
<div class="grid place-items-center h-screen">
<h1 class="text-8xl font-extrabold text-transparent bg-clip-text bg-gradient-to-r from-purple-400 to-pink-600">
Tailwind CSS
</h1>
</div>
@goddoe
goddoe / find_all_span.py
Created November 10, 2022 02:04
find_all_span.py
def find_all_span(d, s):
span_list = []
offset = -1
while True:
start_idx = d.find(s, offset + 1)
if start_idx == -1:
break
@goddoe
goddoe / parallel_apply.py
Last active August 23, 2022 17:05
parallel_apply.py
from functools import partial
import pandas as pd
import numpy as np
def apply(df, func):
return df.apply(func, axis=1)
def parallel_apply(df, func, n_cores):
df_split = np.array_split(df, n_cores)
pool = Pool(n_cores)
@goddoe
goddoe / binary_search.py
Last active July 18, 2022 11:07
iterative binary_search
def binary_search(arr, target):
low, high = 0, len(arr) - 1
while low <= high:
mid = (low+high) // 2
if arr[mid] > target:
high = mid -1
elif arr[mid] == target:
return mid
else:
low = mid + 1
@goddoe
goddoe / ls_to_array.sh
Created June 21, 2022 09:06
ls to array
model_path_list=($(ls -d ${model_ckpt_dir}/model_*))
@goddoe
goddoe / array operations.sh
Created June 21, 2022 08:29
array operations
# for loop array
# Reference: https://www.tutorialkart.com/bash-shell-scripting/bash-append-an-array-to-another-array/
array=( one two three )
for i in "${array[@]}"
do
echo "$i"
done
# append array
# Reference: https://www.tutorialkart.com/bash-shell-scripting/bash-append-an-array-to-another-array/
@goddoe
goddoe / convenient argument input.py
Created June 16, 2022 02:01
convenient argument input
forward_args = inspect.getfullargspec(self.model.forward).args[1:]
x = {k: v for k, vin self.transform(text) if k in forward_args}