Sean Smith sean-smith

DIY Metar Map

Congrats! You've purchased your DIY Metar map kit and now you need to set it up so it connects to your local wifi network and illuminates airports in the correct order.

The kit contains:

Raspberry Pi Zero 2W.
Wiring Harness
WS2811 LED Lights (50 ct)
Power Cord
32 GB Micro SD Card + adaptor

Copy Raspberry ISO on MacOS

based on https://opensource.com/article/21/7/custom-raspberry-pi-image

First image the raspberry pi and make any changes you need via SSH. When you're done make sure to install cockpit:

sudo apt install cockpit

Cross EC2/Hyperpod cluster

In this guide we'll show you how to launch g6e instances in EC2 and connect to a Hyperpod cluster via ssh. These instances will live in the same AZ and mount the same filesystems as the cluster. In this guide we assume you already have a Hyperpod cluster you created by following the workshop content.

We need to launch our g6e instances in the same VPC as your filesystem and in the Public Subnet we created in the initial Cloudformation template so you can SSH directly into the host using the public ip/hostname.

Make sure you're in your Local Environment:

exit # exit the cluster to local environment

Setup Mountpoint CSI driver.

First setup mountpoint following the instructions in the docs.

Steps to setup nvme w/ mountpoint:

Next we'll tell S3 Mountpoint to cache on the 28TB of local NVME available on each P5 instance.

Mount the nvme disks as a single mount - this needs to be done on each p5 instance:

Diagnose GPU Failures on ParallelCluster

To diagnose a node with a bad gpu ip-10-1-69-242 on ParallelCluster, do the following:

Run the nvidia reset command where 0 is the device index shown by nvidia-smi of the gpu you want to reset:

srun -w ip-10-1-69-242 sudo nvidia-smi --gpu-reset -i 0

Run out of EBS space on an ec2 instance?

Make sure the instance has arn:aws:iam::aws:policy/AmazonEC2FullAccess permissions.
Create a script called resize.sh with the following contents:

#!/bin/bash

# Specify the desired volume size in GiB as a command line argument. If not specified, default to 20 GiB.
SIZE=${1:-20}

Install NCCL on a Cluster

To install on the cluster we'll need to install on all nodes in the /opt/nccl directory. In order to do this we'll create a script and then run it on all nodes using the srun command.

Create a script ./install-nccl.sh : and chmod +x install

#!/bin/bash

# install nccl

Install AWS OFI NCCL

Change into the shared directory

cd /fsx

Create a script install-nccl-aws-ofi.sh to install AWS OFI NCCL:

	#!/usr/bin/env python
	# Copyright (c) Facebook, Inc. and its affiliates.
	#
	# This source code is licensed under the MIT license found in the
	# LICENSE file in the root directory of this source tree.
	#


	import os
	import sys

	#!/bin/bash
	# run as root, then validate with:
	# chronyc sources -v
	# chronyc tracking
	# see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html#configure-time-sync

	apt install -y chrony
	sed -i '/\# See http:\/\/www.pool.ntp.org\/join.html for more information./a server 169.254.169.123 prefer iburst minpoll 4 maxpoll 4\npool time.aws.com iburst' /etc/chrony/chrony.conf
	systemctl enable --now chrony
	/etc/init.d/chrony restart