Skip to content

Instantly share code, notes, and snippets.

View kiyoon's full-sized avatar

Kiyoon Kim kiyoon

126 followers · 36 following
View GitHub Profile
@kiyoon
kiyoon / eddie_server.md
Last active August 17, 2021 05:15
Slurm cheatsheet

Node summary

qstat -F gpus,gputype,mem_total -q gpu

Interactive session

qlogin -pe gpu-titanx 1
@kiyoon
kiyoon / human_time_duration.py
Last active June 11, 2021 14:46 — forked from borgstrom/human_time_duration.py
Python: Convert seconds (duration) to human readable string
TIME_DURATION_UNITS = (
('week', 60*60*24*7),
('day', 60*60*24),
('hour', 60*60),
('min', 60),
('sec', 1)
)
def human_time_duration(seconds):
@kiyoon
kiyoon / colour_logging_main.py
Last active October 13, 2021 07:25
Logging with SUCCESS level, and coloured. Global exception catching will log any exception too.
#!/usr/bin/env python3
# pip install coloredlogs verboselogs
import os
import coloredlogs, logging, verboselogs
#logger = logging.getLogger(__name__)
logger = verboselogs.VerboseLogger(__name__) # add logger.success
log_path = 'train.log'
@kiyoon
kiyoon / telegram_post.py
Last active January 1, 2021 06:34
Send Telegram messages (text, photos, and Matplotlib figures).
#!/usr/bin/env python3
# For command line usage, include the token and chat ids in advance in the script.
# If you want to use only the functions, there's no need to include this.
telegram_token = ""
# Get chat id by opening the URL: https://api.telegram.org/bot{token}/getUpdates
telegram_chat_ids = [""]
@kiyoon
kiyoon / ffmpeg_nvidia_conda_install.sh
Last active January 5, 2026 11:43
Install nvidia accelerated ffmpeg in a conda environment.
git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
cd nv-codec-headers
vi Makefile # change the first line to PREFIX = ${CONDA_PREFIX}
make install
cd ..
git clone https://git.ffmpeg.org/ffmpeg.git
cd ffmpeg
git checkout n4.2.2
conda install nasm
@kiyoon
kiyoon / DALI_video_loader.py
Last active February 11, 2020 17:26
PyTorch video loader utilising GPU (CUDA) using NVIDIA DALI > 0.18.
from nvidia.dali.pipeline import Pipeline
from nvidia.dali.plugin import pytorch
import nvidia.dali.ops as ops
import nvidia.dali.types as types
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--file_list', type=str, default='file_list.txt',
help='DALI file_list for VideoReader')
@kiyoon
kiyoon / argparse_example.py
Last active January 30, 2024 13:42
import argparse
from rich_argparse import ArgumentDefaultsRichHelpFormatter
def get_parser():
parser = argparse.ArgumentParser(
description="Example of argparse usage, with default values printed in help.",
formatter_class=ArgumentDefaultsRichHelpFormatter,
)
@kiyoon
kiyoon / video_aug.sh
Created January 11, 2020 22:10
Generate video augmentation data
#!/bin/bash
if [ $# -lt 2 ]
then
echo "usage: $0 [input_dir] [output_dir] [output_resolution=224x224]"
echo "Crops the video into 4 corners and 1 centre, and flips the video horizontally to do the same, resulting in 10 augmentation per video."
exit 1
fi
input_dir="$1"
@kiyoon
kiyoon / docker-compose.yml
Last active February 7, 2020 19:29
Sample docker-compose file
#docker-compose.yml : Make container as described here.
#Author : Hyeonsu Lyu, hslyu@unist.ac.kr, +82 10-5117-9780
# Kiyoon Kim (kiyoon.kim@ed.ac.uk)
#First version, Jan. 30, 2019
version: '2'
services:
aislab-docker:
container_name: base_env
image: kiyoon/docker-for-ML:cuda10.1-cudnn7
@kiyoon
kiyoon / ffmpeg_nvidia_compress_video.sh
Created December 30, 2019 00:39
Compress a video file (specifically produced from Canon M50) using ffmpeg and NVIDIA hardware acceleration. Note that if colour settings are not specified, some players may use a limited colour range and display wrong colour.
ffmpeg -hwaccel cuvid -c:v h264_cuvid -i <input.MP4> -c:v h264_nvenc -rc:v vbr_hq -cq:v 19 -b:v 10000k -maxrate:v 20000k -profile:v high -color_range pc -colorspace bt709 -color_trc bt709 -color_primaries bt709 -c:a copy <output.mp4>