jakebrinkmann/0-DEV-ENV.md

Last active July 11, 2022 22:37

Star (1) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/jakebrinkmann/1c19d93972d156e91c2b4bf7be73910c.js"></script>
Save jakebrinkmann/1c19d93972d156e91c2b4bf7be73910c to your computer and use it in GitHub Desktop.

Download ZIP

Developer Environment -- All my "notes to self" about my developer environment (DEV-ENV)

Raw

0-DEV-ENV.md

👩‍💻 👨‍💻

Hello Friends. This is my space where I want to spread knowledge as far as I can.

Raw

1-OS-MBP.md

MacBook Pro

macOS Monterey - Version 12.4
MacBook Pro (14-inch, 2021)
Chip: Apple M1 Max
Memory: 64 GB
Storage: 2TB SSD

Install Homebrew (https://brew.sh/)
git clone [email protected]:jakebrinkmann/dotfiles.git ~/.dots
ln -s ~/.dots/dots/brew/Brewfile ~/
brew bundle install
cd ~/.dots/dots
stow -t ~ zsh git vim bin bash
Install iTerm2 (https://iterm2.com/)
Install "Command Line Developer Tools" (macOS SDK)
Set Dock to auto-hide
Install oh-my-zsh: https://ohmyz.sh/#install
Install https://github.com/romkatv/powerlevel10k

Open or close the Spotlight window: Command-Space bar

[ ! -f ~/.ssh/id_rsa.pub ] && ssh-keygen -t ecdsa -b 521 -C "[email protected]"
cat .ssh/id_ecdsa.pub | pbcopy
# ... https://github.com/settings/keys

Raw

1-OS-Win10.md

Install WSL

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

wsl --update
wsl --shutdown
wsl --unregister "Ubuntu-20.04"

wsl --list --online
# NAME            FRIENDLY NAME
# Ubuntu-20.04    Ubuntu 20.04 LTS

wsl --install --distribution Ubuntu-20.04

Setup Distro (Ubuntu)

sudo nano /etc/wsl.conf

# Now make it look like this and save the file when you're done:
# [automount]
# root = /
# options = "metadata"

sudo sed --in-place 's/sudo.*ALL=(ALL:ALL) ALL/sudo ALL=(ALL) NOPASSWD:ALL/' /etc/sudoers

PS> wsl --shutdown # turn off all wsl instances such as docker-desktop

PS> notepad.exe "$env:USERPROFILE/.wslconfig"  # Make sure this is saved as UTF-8
PS> cat .\.wslconfig
[wsl2]
processors=2  # Makes the WSL 2 VM use two virtual processors
memory=8GB  # Limits VM memory in WSL 2 to 4 GB
swap=16GB
localhostForwarding=true

PS> wsl
$ free -h
$ cat /proc/sys/fs/file-max

# Increase the Max Open File Limit
$ cat /etc/security/limits.conf
* hard nofile 100000
* soft nofile 100000
session required pam_limits.so

sudo apt update
sudo apt upgrade --assume-yes
sudo apt-get --fix-missing --assume-yes install build-essential procps curl file git stow

[ ! -f ~/.ssh/id_rsa.pub ] && ssh-keygen -t rsa -b 4096 -C "[email protected]"
cat ~/.ssh/id_rsa.pub | clip.exe

# ... https://github.com/settings/keys
# ... https://gitlab.com/-/profile/keys

git clone [email protected]:jakebrinkmann/dotfiles.git ~/.dots

cd ~/.dots/dots
stow -t ~ bash git vim bin

cd ~/.dots/install
bash brew.sh

Make CAPSLOCK trigger a CTRL instead

$ cat /c/Users/jbrinkma/Desktop/keys.reg
Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout]
"Scancode Map"=hex:00,00,00,00,00,00,00,00,02,00,00,00,1d,00,3a,00,00,00,00,00

Raw

2-TOOLS-Docker.md

Install Docker for Windows
~~Guide to Setting Up Docker for Windows and WSL to Work Flawlessly~~
Install Docker-CE on WSL Ubuntu
Powershell Admin: Enable-WindowsOptionalFeature -Online -FeatureName $("VirtualMachinePlatform", "Microsoft-Windows-Subsystem-Linux")
Download: https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi
wsl --set-default-version 2
https://docs.docker.com/docker-for-windows/wsl/
DO NOT DO THIS ON WSL2
~~cd ~/.dots; sudo ./install/base.sh; sudo ./install/docker.sh~~
~~sudo usermod -aG docker $USER~~
~~echo "export DOCKER_HOST=tcp://127.0.0.1:2375" >> ~/.bashrc && source ~/.bashrc~~

Summary

To develop Linux software on a machine running Windows, historically VirtualMachines are most used. This creates some headaches and overhead on the host system that aren't "seamless".
Bash on Ubuntu on Windows (or, the Windows Subsystem for Linux) offers native system call mappings in order to remove the appearance of any seams between Windows/Linux development.
The "Seamless" aspect is not quite there Out of the Box, but with a little guidance we can get pretty close to developing on a Windows machine with all the benefits of a Linux environment.
Docker presents a unique challenge, because containers run inside their own container network, and require some special attention

Alternatives

Virtual Machines: Windows 10 Enterprise comes with Hyper-V, capable of running VirtualMachines. It will require proper NAT configuration to get to the internet, and other file-sharing config to add the ability to access e.g. a USB stick from the host machine. Unless using a manager like Vagrant, this process is largely procedural and isn't easily reproducible via automation tooling.
Cygwin: Instead of using native Linux tools, the Cygwin compiler can be used to create what appears like a Unix environment while running on Windows. However, it requires us to re-compile all our favorite tools using our compiler, a task which for some large projects becomes an ever expanding problem as you search for all the dependencies required by a tool.

Raw

2-TOOLS-Git.md

access to code repos

Gitlab/Github account
Add SSH keys

Raw

2-TOOLS-Python.md

Virtualenv

https://virtualenv.pypa.io/en/latest/userguide/#making-environments-relocatable

For local development, create a python virtual environment:

cd resources
virtualenv -p python3.7 venv
virtualenv --relocatable venv

Activate the environment

source ./venv/bin/activate

NOTE: Always activate this environment.

https://martin-thoma.com/python-style-guide/

PDB Cheat Sheet

Jupyter Lab

source venv/bin/activate
python -m pip install ipykernel
MYENV=sandbox; python -m ipykernel install --user --name $MYENV --display-name "Python ($MYENV)"
# Installed kernelspec myenv in /c/Users/USERNAME/WSL/.local/share/jupyter/kernels/myenv

# add a custom path to always import from
echo '' >> venv/lib/python3.7/site-packages/MY_CUSTOM_PYTHON_PATH.pth

# Allow reloading imports inside a notebook (without restarting the kernel)
c.InteractiveShellApp.extensions = ['autoreload']
c.InteractiveShellApp.exec_lines = ['%autoreload 2']

Ubuntu LTS 18.04

How to Install Python 3.7 on Ubuntu 18.04

Testing & Linting

pip install pytest pytest-cov
pip install pylint pycodestyle pydocstyle black

VSCode Pytest Import Errors

Raw

3-TECH-PostgreSQL.md

Example to Connect to Postgres from Windows
Provide password using the password file
sudo apt install postgresql-client-common
sudo apt -y install postgresql-client-12

Read table descriptions across a schema into a CSV file:

psql -h localhost -U cca > ~/Desktop/table_descriptions.csv <<__END__
SELECT table_schema || '.' || table_name as table_name, obj_description(pg_class.oid) 
FROM pg_class 
JOIN information_schema.tables
ON information_schema.tables.table_name = pg_class.relname

WHERE pg_class.relkind='r' 
AND pg_class.relname !~ 'pg_'
AND table_schema NOT IN ('information_schema');
__END__

$ cat schemaspy.properties
schemaspy.t=pgsql
schemaspy.dp=postgresql-9.4.1208.jre6.jar
schemaspy.host=127.0.0.1
schemaspy.port=5432
schemaspy.db=database
schemaspy.u=username
schemaspy.p=password
schemaspy.o=output

$ cat script.sh
java -jar schemaspy-6.1.0.jar -vizjs -all -norows -rankdirbug

$ ls *.jar
postgresql-9.4.1208.jre6.jar*  schemaspy-6.1.0.jar*