Skip to content

Instantly share code, notes, and snippets.

View nanguoyu's full-sized avatar

Dong Wang nanguoyu

View GitHub Profile
@nanguoyu
nanguoyu / weigt_and_bias.py
Last active November 5, 2024 13:42
weigt and bias
import wandb
model_profile_str = f'{args.model}_{args.wider_factor}Xwider_{args.dataset}'
wandb_proiject_name = "model folding"
experiment_name = f"train_{model_profile_str}"
run = wandb.init(project=wandb_proiject_name, name=experiment_name, entity="naguoyu",
config={"dataset":args.dataset},
)
@nanguoyu
nanguoyu / fast_torchvision_dataloader.py
Last active July 18, 2024 12:16
Fast image dataloader for Pytorch models
"""
@File : fast_torchvision_dataloader.py
@Author: Dong Wang
@Date : 2024/06/25
@Description : a fast image dataloader for Pytorch models. It tries to use FFCV to speed up your dataloader for vision tasks.
You need first install FFCV in your Python ENV and run prepare_ffcv_dataset.py to prepare datasets in FFCV.
"""
import os
from torch.utils.data import DataLoader
# Go to https://docs.conda.io/en/latest/miniconda.html and choose the suitable file link
wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.12.0-Linux-x86_64.sh
# install Miniconda
bash Miniconda3-py37_4.12.0-Linux-x86_64.sh
conda create -n env_name python=3.7
conda activate env_name
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
panic(cpu 1 caller 0xfffffe0026801790): watchdog timeout: no checkins from watchdogd in 93 seconds (127 total checkins since monitoring last enabled)
Debugger message: panic
Memory ID: 0x6
OS release type: User
OS version: 22C65
Kernel version: Darwin Kernel Version 22.2.0: Fri Nov 11 02:04:44 PST 2022; root:xnu-8792.61.2~4/RELEASE_ARM64_T8103
Fileset Kernelcache UUID: ADB288150AFF2B26A022D2179A26F30C
Kernel UUID: D43063DF-7FAB-3E39-9807-2FC6A0C7F76A
Boot session UUID: BD64D3CE-D999-44A9-A66B-88DAF57DAA20
iBoot version: iBoot-8419.60.44
@nanguoyu
nanguoyu / 3D_Beta_Space.py
Last active December 12, 2022 10:24
3D Beta space
hyper_output = []
with tqdm(range(-180, 181, 10), position=0) as t:
for x in t:
for y in tqdm(range(-180, 181, 10), disable=True):
for z in tqdm(range(-180, 181, 10), disable=True):
angles = torch.tensor([x,y,z])/180*torch.pi
Hyper_x = transform_angles(angles=angles).to(device=gpu_computation)
hyper_output.append(model.hyper_stack(Hyper_x).cpu().detach().numpy())
git clone https://github.com/nanguoyu/GenoCAE.git
cd GenoCAE/
docker build -t gcae/genocae:build -f docker/build.dockerfile .
docker run -it --rm -v ${PWD}:/workspace gcae/genocae:build python3 run_gcae.py --help
@nanguoyu
nanguoyu / Ubuntu18.04 Python3.8
Last active June 6, 2021 21:03
Install Python3.8 in Ubutun18.04 from source
apt-get update && apt-get upgrade -y &&\
apt-get install -y wget \
build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev
wget https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tgz &&\
tar -xf Python-3.8.0.tgz &&\
cd Python-3.8.0 &&\
./configure --enable-optimizations &&\
make -j8 &&\
@nanguoyu
nanguoyu / OpenCV_GPU.sh
Last active March 4, 2021 23:51
Install OpenCV with GPU support
sudo apt-get install qt5-default
sudo apt-get install opencl-headers
sudo pip install pylint
# Install minimal prerequisites (Ubuntu 18.04 as reference)
sudo apt update && sudo apt install -y cmake g++ wget unzip
# Download and unpack sources
wget -O opencv.zip https://github.com/opencv/opencv/archive/master.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/master.zip
unzip opencv.zip
unzip opencv_contrib.zip
@nanguoyu
nanguoyu / coco.sh
Last active February 9, 2021 20:57 — forked from mkocabas/coco.sh
Download COCO dataset. Run under 'datasets' directory.
mkdir coco
cd coco
mkdir images
cd images
wget -c http://images.cocodataset.org/zips/train2017.zip
wget -c http://images.cocodataset.org/zips/val2017.zip
wget -c http://images.cocodataset.org/zips/test2017.zip
wget -c http://images.cocodataset.org/zips/unlabeled2017.zip