Yingtao Tian alantian

alantian / lsut_text_preprocess.py

Created November 9, 2018 13:29

(tentative) text preprocess for JSUT dataset

	#!/usr/bin/env python3

	from collections import Counter
	from os.path import join
	import re
	import os

	import pykakasi

	# --- Routines

alantian / CompileTF.sh

Last active October 29, 2018 22:18

Compile TensorFlow 1.12 with CUDA 10 on conda environment on Arch Linux.

	# Compiling TensorFlow 1.12 with CUDA 10 on conda environment on Arch Linux.
	#
	# Desired versions:
	# - TensorFlow 1.12
	# - Python 3.6
	#
	# Tested on Arch Linux with:
	# - cuda 10.0.130-2
	# - cudnn 7.3.0-1
	# - nccl 2.3.4.4-2

alantian / run_batch.py

Created September 22, 2018 18:48

	#!/usr/bin/env python3
	"""Run multiple jobs, potentially in parallel to leverage multiple GPUs on host."""

	import hashlib
	from os import environ
	from os.path import expanduser, exists
	import subprocess
	import shlex
	import time

alantian / tensorboard_report.py

Created August 1, 2018 06:06

Chainer Extension that reports scalars to tensorboard

	import os
	import sys

	from chainer.training import extension
	from chainer.training.extensions import log_report as log_report_module
	from chainer.training.extensions import util

	import tensorflow as tf

alantian / Dockerfile

Last active April 26, 2018 00:09

Dockerfile for running code built on some common deep learning frameworks.

	FROM nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04

	#-------------------------------------------------------------------------------
	### Enable UTF8 in docker instance
	#-------------------------------------------------------------------------------
	RUN apt-get update -y && \
	apt-get install -y locales && \
	rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*
	RUN locale-gen en_US.UTF-8
	ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'

alantian / make_dataset.py

Created November 7, 2017 23:21

make dataset for official progressive gan code

	#!/usr/bin/env python2

	import os
	import numpy as np
	import h5py
	import threading
	import Queue
	from PIL import Image
	from tqdm import tqdm
	from joblib import Parallel, delayed

alantian / binary-tag-stat.py

Last active August 16, 2017 04:32

	import pickle
	import pandas
	import json
	from collections import Counter
	import os
	from os.path import join
	import shutil

	games = pickle.load(open('../../data/game_face_170701/games.pickle', 'rb'), encoding='latin1')
	tags = pickle.load(open('../../data/game_face_170701/tags.pickle', 'rb'), encoding='latin1')

alantian / convert_lmdb.py

Created July 16, 2017 12:18

Convert dataset to lmdb in parallel

alantian / extract.py

Created June 21, 2017 06:17

extract dialogue.

	import os
	import re

	from joblib import Parallel, delayed

	def go_lines(lines):

	flow = []
	names = []

alantian / convert-to-utf8.ipynb

Created June 20, 2017 07:14

convert files to UTF-8, recursively.

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.