Reed Slaby, March 2018
Having grown tired of wasting a perfectly good monitor on my ethereum mining rig, I finally decided to replace the Ubuntu Desktop 16.04 installation with Ubuntu Server 16.04. Many of the gtutorials available at the time of this writing range from unecessarily complicated to flat-out wrong. This guide is intended to address many of those shortfalls.
To complete this guide, you should already have:
- Basic knowledge of Linux
- An Ethereum wallet
- ASUS Z270-P Motherboard
- Intel Celeron G3930 Processor
- 8GB DDR4 RAM
- 128GB HDD
- EVGA SuperNOVA G3 750W ATX PSU
- 2x NVIDIA GTX 1060
- 3x NVIDIA GTX 1070
- Monitor and Keyboard (for initial installation)
I downloaded a copy of Ubuntu Server 16.04 (amd64) and copied it to a USB flash drive using dd.
The hard drive was partitioned as follows:
Partition 0: 256MB - EFI
Partition 1: 8GB - swap
Partition 2: 40GB - /
Partition 3: 30GB - /home
Note: There is plenty of room for optimization in this partitioning scheme. Only about ~3GB of the root partition will be used in the end. Reduce at your own risk!
If you receive an error that a file system could not be created on the EFI partition, try deleting and recreating the partition. Also ensure that no other EFI hard drives are connected.
The first order of business after logging in the first time is to update and install SSH:
sudo apt update
sudo apt upgrade
sudo apt install openssh-server
Take note of the IP address:
ip addr show
At this point, the monitor and keyboard can be disconnected. I then connected over SSH to ensure that I was not inadvertently relying on a connected monitor in any way while performing the rest of the setup.
These utilities will be required to install and run the NVIDIA driver/utilities:
sudo apt install build-essential libgtk-3-0 xorg
Go to the NVIDIA website and download the latest Linux 64-bit driver.
Extract the archive and execute the installation script:
gzip -d NVIDIA-Linux-x86_64-xxx.xx.tar.gz
tar -xvf NVIDIA-Linux-x86_64-xxx.xx.tar
chmod +x NVIDIA-Linux-x86_64-xxx.xx.run
sudo ./NVIDIA-Linux-x86_64-xxx.xx.run
Feel free to answer 'Yes' to running nvidia-xconfig, though it will probably only pick up the first card at this time.
Run nvidia-smi to ensure that all the GPUs are visible. If some or all are missing, stop and troubleshoot.
Reboot the machine:
sudo init 6
Run nvidia-xconfig to enable overclocking and allow things to work without a real monitor connected:
sudo nvidia-xconfig --cool-bits=31 --allow-empty-initial-configuration
Check the /etc/X11/xorg.conf file to ensure all GPUs have been configured. Each GPU should have a Device, Monitor, and Screen section. The Screens should show the following options:
Section "Screen"
Identifier "ScreenX"
Device "DeviceX"
Monitor "MonitorX"
DefaultDepth 24
**Option "AllowEmptyInitialConfiguration" "True"
Option "Coolbits" "31"**
SubSection "Display"
Depth 24
EndSubSection
EndSection
Annoyingly enough, you will need to start an X server in order to overclock the GPUs. Doing this also requires root privileges, so I updated my sudoers file to slightly ease the pain.
Using visudo, I added the following lines to sudoer:
USERNAME HOST = (root) NOPASSWD: /usr/bin/xinit
USERNAME HOST = (root) NOPASSWD: /usr/bin/nvidia-smi
Note: replace USERNAME and HOST with your specific info. For example, joe@glados would use "joe glados =..."
Start an X server and push it into the background:
sudo xinit &
export DISPLAY=:0.0
Enable persistence mode on the GPUs:
sudo nvidia-smi -pm 1
Helpful overclocking commands:
// Set GPU 0 power level to 100W
sudo nvidia-smi -i 0 -pl 100
// Enable GPU 0 manual fan control and set to 50%
nvidia-settings -a '[gpu:0]/GPUFanControlState=1' -a '[fan:0]/GPUTargetFanSpeed=50'
// Increase GPU 0 memory clock by 1000Hz
nvidia-settings -a '[gpu:0]/GPUMemoryTransferRateOffset[3]=1000'
TODO