Yucheng Zhao yuzhms

🤗Huggingface Model Downloader

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
🚫 File Exclusion: Use --exclude or --include to skip or specify files, save time for models with duplicate formats (e.g., *.bin or *.safetensors).
🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
🪞 Mirror Site Support: Set up with HF_ENDPOINT environment variable.

The new 1.10 version of hdf5 library is stable. You can now install from the source to use this new capability.

The installation process has a few caveats, so here is how to do it.

Note:

you need to first remove the hdf5 libraries currently installed in ubuntu.
Then you need to reinstall h5py.
when you reinstall h5py via pip, use the --no-binary flag. This is because the h5py wheel comes with its own hdf5 binary.

Instructions:

ffmpeg

Converting Audio into Different Formats / Sample Rates

Minimal example: transcode from MP3 to WMA:
ffmpeg -i input.mp3 output.wma

You can get the list of supported formats with:
ffmpeg -formats

Convert WAV to MP3, mix down to mono (use 1 audio channel), set bit rate to 64 kbps and sample rate to 22050 Hz:

In this article, I will share some of my experience on installing NVIDIA driver and CUDA on Linux OS. Here I mainly use Ubuntu as example. Comments for CentOS/Fedora are also provided as much as I can.

	from __future__ import print_function

	import argparse
	import torch.backends.cudnn as cudnn
	import torch.nn.functional as F
	import torch.optim as optim
	import torch.utils.data.distributed
	from torchvision import models
	import horovod.torch as hvd
	import timeit

	#!/usr/bin/env python
	# -- coding:UTF-8 --

	import torch
	import torch.nn as nn
	import torch.nn.init as init


	def weight_init(m):
	'''

	# -- coding: utf-8 --

	from __future__ import absolute_import
	from __future__ import division
	from __future__ import print_function

	import argparse
	import numpy as np
	import os
	import random