This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import torch | |
| import torch.nn as nn | |
| import torch.optim as optim | |
| import torch.distributed as dist | |
| import torch.multiprocessing as mp | |
| from torch.utils.data import DataLoader | |
| from torch.utils.data.distributed import DistributedSampler | |
| # Define the sentiment analysis model | |
| class SentimentAnalysisModel(nn.Module): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """A naive imnplementation for interactively cloning private python repositories in google colab""" | |
| from google.colab import drive | |
| def get_config(path: str): | |
| """deserialised configuration from disk or return an empty dict""" | |
| if not os.path.exists(path): | |
| return {} | |
| with open(path) as cbf: | |
| _colab_cfg = cbf.read() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## Proble: | |
| # Apparently some blobs were not uploaded to the server | |
| # Causeing git lfs to stop working | |
| # I could not even push to other remotes becuase of this | |
| # Solution 1: | |
| # Revert to a commit before running: git lfs install | |
| git checkout <commit-hash> | |
| git switch -n mynewbranch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pytest | |
| """ | |
| References: | |
| =========== | |
| - https://www.freblogg.com/pytest-functions-mocking-1 | |
| """ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """Given a file containing a list of places, get the well formatted address containing the country, zip code, iso names etc. | |
| using the Google Maps API | |
| """ | |
| import os | |
| import json | |
| import logging | |
| import traceback | |
| import requests | |
| import pickle | |
| import pycountry |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Most of the commands are based on this tutorial: https://www.sitepoint.com/wsl2/ | |
| # Step 1: Windows pre-requisites | |
| # Official guide: https://docs.microsoft.com/en-us/windows/wsl/install-manual | |
| # Run the following commands in Powershell(as admin) | |
| dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart | |
| dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart | |
| # Download and Run: https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi | |
| wsl --set-default-version 2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Install cron to sun scheduled scripts | |
| sudo apt update | |
| sudo apt install cron | |
| sudo systemctl enable cron | |
| # Add you config in crontab | |
| crontab -e | |
| # Syntax (Reference: https://www.digitalocean.com/community/tutorials/how-to-use-cron-to-automate-tasks-ubuntu-1804) | |
| # minute hour dayofMonth month dayOfWeek <you-commands> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| IMAGE_TAG=ecr_repo_name | |
| ECR_URL=public.ecr.aws/<your-ec-id> | |
| REGION=east-us-1 | |
| # setup aws cli | |
| # Reference: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html | |
| curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Reference: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/expand-disks | |
| # Accoring to the documentation, this is only supported for data disks only | |
| # At the time of this writing, this feature is still experimental | |
| # i. Opt-in as follows: | |
| az feature register --namespace Microsoft.Compute --name LiveResize | |
| RG=myResourceGroup | |
| VM=myVmName |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # Scipt tested on Ubuntu 18.04 | |
| # i. Make sure java is installed | |
| # If you intend to use hive, use java-8 | |
| sudo apt-get update -y | |
| sudo apt-get install openjdk-8-jdk -y | |
| # 2. Now we can install hadoop | |
| cd /tmp |