Duoduoyuan duoduoyeah

Setting up a GPU server with scheduling and containers

Our group recently acquired a new server to do some deep learning: a SuperMicro 4029GP-TRT2, stuffed with 8x NVidia RTX 2080 Ti. Though maybe a bit overpowered, with upcoming networks like BigGAN and fully 3D networks, as well as students joining our group, this machine will be used quite a lot in the future.

One challenge is, is how to manage these GPUs. There are many approaches, but given that most PhD candidates aren't sysadmins, these range from 'free-for-all', leading to one person hogging all GPUs for weeks due to a bug in the code, to Excel sheets that noone understands and noone adheres to because changing GPU ids in code is hard. This leads to a lot of frustration, low productivity and under-utilisation of these expensive servers. Another issue is conflicting software versions. TensorFlow and Keras, for example, tend to do breaking API changes every now and then. As t

Connect to a Google Cloud Compute Engine using Mosh (mobile shell)

Mosh is a great way to overcome lag and flakey connection issues when accessing remote terminals over SSH: https://mosh.org/

Unfortunately, there's still not a lot of clear documentation for "idiots" because a lot of the existing guides assume a) "it just works", or b) you're already skilled enough to set everything up from scratch (assumed knowledge/use cases).

One of the problems I ran into for quite a long time yesterday was getting the darn thing to connect to my Google Cloud instance. I kept running into the mosh: Nothing received from server on UDP port 60001. [To quit: Ctrl-^ .] error (by the way - to quit, you have to press . after Ctrl-^ - it's not just Ctrl-^, there is a dot there too!)

FAQs and the like made vague spartan references to NATs, firewalls, iptables and the like and I tried out various commands and configurations on the se

	import math
	import requests

	from urllib.parse import quote
	from urllib.request import urlopen

	import xml.etree.ElementTree as ET
	import json

	class XPLORE: