padeoe

🤗Huggingface Model Downloader

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
🚫 File Exclusion: Use --exclude or --include to skip or specify files, save time for models with duplicate formats (e.g., *.bin or *.safetensors).
🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
🪞 Mirror Site Support: Set up with HF_ENDPOINT environment variable.

DDNS for Domain on Name.com

NAME.COM provide the API to create/update DNS record, we can implement DDNS without depending on any third party DDNS service.

Python 3.8+ (because I used the latest := operator in 3.8, if you use 3.6 or others, you can modify the codes, very easy)
v4 REST API of name.com
Use ipinfo.io to get public IP

Step 1: Prepare a Proxy (Static IP)

Since we would like to setup DDNS, which means our server doesn't have a fixed IP, but the API of name.com requires the caller's IP whitelisted in advance, which means the caller's IP need to be fixed or limited in the number. However, the dynamic from ISP is not stable. So, a proxy server is required to solve the problem.

功能特性

每隔 5 分钟执行一次网络系统登录检测，检查当前是否在线，如果不在线则登录上线。

定时检测
保存登录日志
用户名密码参数化

依赖 Python 3.

1. 创建登陆脚本

For typical use cases, I prefer using nvitop to view detailed information. This script offers a dependency-free implementation.

The script enhances the functionality of nvidia-smi and provides the following info:

Username
full process Command
GPU ID
PID

This is useful on multi-user servers and can be used to quickly identify which user is using the GPU and running what kind of program.

[Script] Auto-Renew Wildcard Letsencrypt Certs for name.com

This tutorial can Auto-Renew wildcard letsencrypt certs for domain on name.com.

wildcard domain, e.g., *.example.com
domain hosted by https://www.name.com, we use the API of name.com to add DNS TXT records with a shell script.

Wildcard certificates issued by letsencrypt.org need DNS TXT record to challenge, we can add TXT record manually when you apply the cert. If we want to automate it, we need to write a script that use the API of DNS provider to add TXT record. certbot has provided command argument --manual-auth-hook to pass the script.

Step 1: Get a API token

Get your own api token provided by name.com: https://www.name.com/account/settings/api.

	{
	"domain": [
	"domain:nju.edu.cn",
	"domain:njuftp.org",
	"domain:ingentaconnect.com",
	"domain:nutrition.org",
	"domain:geoscienceworld.org",
	"domain:westlaw.com",
	"domain:agu.org",
	"domain:emeraldgrouppublishing.com",

	FROM pytorch/pytorch:1.1.0-cuda10.0-cudnn7.5-devel

	# use local apt mirror
	RUN sed -i 's/archive.ubuntu.com/mirrors.aliyun.com/g;s/security.ubuntu.com/mirrors.aliyun.com/g' /etc/apt/sources.list
	RUN apt-get update

	# use local pip mirror
	RUN mkdir ~/.pip \
	&& printf '%s\n%s\n%s\n' '[global]' 'trusted-host = mirrors.aliyun.com' \
	'index-url = https://mirrors.aliyun.com/pypi/simple'>> ~/.pip/pip.conf

	# pull the latest image
	check_update=`docker pull gitlab/gitlab-ce:latest`

	# Check if newer image pulled
	if [[ $check_update == "Image is up to date for gitlab/gitlab-ce:latest" ]]
	then
	echo 'Gitlab is already up to date!'
	else
	echo 'Start to update Gitlab...'

	FROM tensorflow/tensorflow:latest-gpu-py3

	# 修改apt源
	RUN sed -i 's/archive.ubuntu.com/mirrors.aliyun.com/g;s/security.ubuntu.com/mirrors.aliyun.com/g' /etc/apt/sources.list
	RUN apt-get update

	# 修改pip源
	RUN mkdir ~/.pip \
	&& printf '%s\n%s\n%s\n' '[global]' 'trusted-host = mirrors.aliyun.com' \
	'index-url = https://mirrors.aliyun.com/pypi/simple'>> ~/.pip/pip.conf

	//含"无用比较"的插入排序
	void insertion_sort(int a[], int n) {
	for (int i = 0; i < n; i++) {
	for (int j = 0; j < i; j++) {
	if (a[i] < a[j]) {
	int tmp = a[j];
	a[j] = a[i];
	a[i] = tmp;
	}
	}