Skip to content

Instantly share code, notes, and snippets.

View ndamulelonemakh's full-sized avatar
🍸
Solution explorer

Ndamulelo Nemakhavhani ndamulelonemakh

🍸
Solution explorer
View GitHub Profile
@ndamulelonemakh
ndamulelonemakh / pytorch-basic.py
Last active May 5, 2023 13:10
Distribute training samples
import torch
import torch.nn as nn
import torch.optim as optim
import torch.distributed as dist
import torch.multiprocessing as mp
from torch.utils.data import DataLoader
from torch.utils.data.distributed import DistributedSampler
# Define the sentiment analysis model
class SentimentAnalysisModel(nn.Module):
@ndamulelonemakh
ndamulelonemakh / colab_clone_private_repo.py
Last active August 10, 2022 01:42
Google colab utils
"""A naive imnplementation for interactively cloning private python repositories in google colab"""
from google.colab import drive
def get_config(path: str):
"""deserialised configuration from disk or return an empty dict"""
if not os.path.exists(path):
return {}
with open(path) as cbf:
_colab_cfg = cbf.read()
## Proble:
# Apparently some blobs were not uploaded to the server
# Causeing git lfs to stop working
# I could not even push to other remotes becuase of this
# Solution 1:
# Revert to a commit before running: git lfs install
git checkout <commit-hash>
git switch -n mynewbranch
@ndamulelonemakh
ndamulelonemakh / pytest_mocking_cheat_sheet.py
Created July 8, 2022 11:31
Misc Unit testing cheat sheets
import pytest
"""
References:
===========
- https://www.freblogg.com/pytest-functions-mocking-1
"""
@ndamulelonemakh
ndamulelonemakh / googlemaps_autoformat_address.py
Created June 29, 2022 10:21
Get auto formatted address from google maps
"""Given a file containing a list of places, get the well formatted address containing the country, zip code, iso names etc.
using the Google Maps API
"""
import os
import json
import logging
import traceback
import requests
import pickle
import pycountry
@ndamulelonemakh
ndamulelonemakh / wsl_for_windows11setup.ps
Last active June 23, 2022 01:06
WSL2 setup on a new Windows 11 PC
# Most of the commands are based on this tutorial: https://www.sitepoint.com/wsl2/
# Step 1: Windows pre-requisites
# Official guide: https://docs.microsoft.com/en-us/windows/wsl/install-manual
# Run the following commands in Powershell(as admin)
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
# Download and Run: https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi
wsl --set-default-version 2
@ndamulelonemakh
ndamulelonemakh / cron_setup.sh
Last active May 31, 2022 12:54
Misc scripts for unix admin
# Install cron to sun scheduled scripts
sudo apt update
sudo apt install cron
sudo systemctl enable cron
# Add you config in crontab
crontab -e
# Syntax (Reference: https://www.digitalocean.com/community/tutorials/how-to-use-cron-to-automate-tasks-ubuntu-1804)
# minute hour dayofMonth month dayOfWeek <you-commands>
@ndamulelonemakh
ndamulelonemakh / push2_aws_ecr.sh
Created May 27, 2022 10:35
Misc scripts for building and publishing containers
#!/bin/bash
IMAGE_TAG=ecr_repo_name
ECR_URL=public.ecr.aws/<your-ec-id>
REGION=east-us-1
# setup aws cli
# Reference: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
@ndamulelonemakh
ndamulelonemakh / az_live_disk_resize.sh
Last active May 31, 2022 06:58
Misc config scripts for Azure
# Reference: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/expand-disks
# Accoring to the documentation, this is only supported for data disks only
# At the time of this writing, this feature is still experimental
# i. Opt-in as follows:
az feature register --namespace Microsoft.Compute --name LiveResize
RG=myResourceGroup
VM=myVmName
@ndamulelonemakh
ndamulelonemakh / install-hadoop.sh
Last active May 26, 2022 00:17
Miscellaneous scripts for installing big data tools for learning workloads
#!/bin/bash
# Scipt tested on Ubuntu 18.04
# i. Make sure java is installed
# If you intend to use hive, use java-8
sudo apt-get update -y
sudo apt-get install openjdk-8-jdk -y
# 2. Now we can install hadoop
cd /tmp