Skip to content

Instantly share code, notes, and snippets.

View ndamulelonemakh's full-sized avatar
🍸
Solution explorer

Ndamulelo Nemakhavhani ndamulelonemakh

🍸
Solution explorer
View GitHub Profile
@ndamulelonemakh
ndamulelonemakh / twitter-scraping-twint-examples.sh
Last active October 8, 2021 10:30
Misc web scraping scripts
# i. Search for tweets containing hastag '#messi' posted in 2020
# Results will be saved to a csv file named 'messitweets.csv'
# Other filters: Only get tweets in english(en), stats: show the number of likes, retweets for each post, count: show the number of posts returned
twint --search "#messi" --since 2020-01-01 --until 2020-12-31 --limit 50000 --popular --csv -o "messitweets.csv" --lang en --stats --hashtags --stats --count
# ii. Collect all tweets from a users timeline
twint --since 2020-01-01 --until 2020-12-31 --limit 50000 --popular --csv -o "out.csv" --lang en --stats --hashtags --stats --count -u endeesa -tl
@ndamulelonemakh
ndamulelonemakh / azure-react-staticwebstites-deploy.sh
Last active May 27, 2022 10:01
Azure deployment scripts for dev and small-scale production workloads
#! bin/bash
# Helper script to deploy the production optimised build of a react app to Azure
# The environment variables below can be changed as required
ResourceGroupName="my-example-rg"
Location="westus"
StorageAccountName="myspecialstorageaccount"
StorageSKU="Standard_LRS"
StorageKind="StorageV2" # Allowed values: BlobStorage, BlockBlobStorage, FileStorage, Storage, StorageV2
StorageContainerName="mysitecontainer01"
@ndamulelonemakh
ndamulelonemakh / create-az-app-getway.sh
Last active January 13, 2022 19:52
Implementing web app resilience in Azure
#! /bin/bash
# Pre-requisits
# Active Azure Subscription
# A virtual network resource
# 2 or more vms serving the same web app
# Reference: https://docs.microsoft.com/en-us/learn/modules/load-balance-web-traffic-with-application-gateway/5-exercise-create-configure-application-gateway
# i.Step 1: Create a gateway subnet
@ndamulelonemakh
ndamulelonemakh / install-hadoop.sh
Last active May 26, 2022 00:17
Miscellaneous scripts for installing big data tools for learning workloads
#!/bin/bash
# Scipt tested on Ubuntu 18.04
# i. Make sure java is installed
# If you intend to use hive, use java-8
sudo apt-get update -y
sudo apt-get install openjdk-8-jdk -y
# 2. Now we can install hadoop
cd /tmp
@ndamulelonemakh
ndamulelonemakh / az_live_disk_resize.sh
Last active May 31, 2022 06:58
Misc config scripts for Azure
# Reference: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/expand-disks
# Accoring to the documentation, this is only supported for data disks only
# At the time of this writing, this feature is still experimental
# i. Opt-in as follows:
az feature register --namespace Microsoft.Compute --name LiveResize
RG=myResourceGroup
VM=myVmName
@ndamulelonemakh
ndamulelonemakh / push2_aws_ecr.sh
Created May 27, 2022 10:35
Misc scripts for building and publishing containers
#!/bin/bash
IMAGE_TAG=ecr_repo_name
ECR_URL=public.ecr.aws/<your-ec-id>
REGION=east-us-1
# setup aws cli
# Reference: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
@ndamulelonemakh
ndamulelonemakh / cron_setup.sh
Last active May 31, 2022 12:54
Misc scripts for unix admin
# Install cron to sun scheduled scripts
sudo apt update
sudo apt install cron
sudo systemctl enable cron
# Add you config in crontab
crontab -e
# Syntax (Reference: https://www.digitalocean.com/community/tutorials/how-to-use-cron-to-automate-tasks-ubuntu-1804)
# minute hour dayofMonth month dayOfWeek <you-commands>
@ndamulelonemakh
ndamulelonemakh / wsl_for_windows11setup.ps
Last active June 23, 2022 01:06
WSL2 setup on a new Windows 11 PC
# Most of the commands are based on this tutorial: https://www.sitepoint.com/wsl2/
# Step 1: Windows pre-requisites
# Official guide: https://docs.microsoft.com/en-us/windows/wsl/install-manual
# Run the following commands in Powershell(as admin)
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
# Download and Run: https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi
wsl --set-default-version 2
@ndamulelonemakh
ndamulelonemakh / googlemaps_autoformat_address.py
Created June 29, 2022 10:21
Get auto formatted address from google maps
"""Given a file containing a list of places, get the well formatted address containing the country, zip code, iso names etc.
using the Google Maps API
"""
import os
import json
import logging
import traceback
import requests
import pickle
import pycountry
@ndamulelonemakh
ndamulelonemakh / pytest_mocking_cheat_sheet.py
Created July 8, 2022 11:31
Misc Unit testing cheat sheets
import pytest
"""
References:
===========
- https://www.freblogg.com/pytest-functions-mocking-1
"""