Pyinfra vs Shell over SSH (also Fabric, Ansible).

pyinfra_vs_bash.md

Overview (tl;dr:)

• For local dev, shell scripts are pretty much always preferred. See: https://github.com/gnat/doit
  • Because 1:1 native equivalent with what is actually run on the system.
• For inventory management, pyinfra starts paying off.

Details

Shell + SSH advantages:

• Shines because 1:1 equivalent with what is actually run on the system.
• A few orders of magnitude less code.
• Shell scripts are preferred for local dev setup.
• Instant ad-hoc management using ssh.sh
• If you feel the need to rewrite in Python, you're probably over-engineering.
• Editors are much better equipped to syntax highlight pure bash than "bash in python strings".

Shell + SSH disadvantages:

• Need to input password for sudo once per run if your script requires sudo.
  • On the bright side, it is not stored anywhere or viewable in ps -aux like in Pyinfra.
• Seperate commands for serial or parallel execution. May be an issue with a lot of servers.
• Doesn't always stop on fail with child scripts.
• Debugging sucks more.

Shell + SSH Notes:

• 🙄 Re-sync can be automated by rsync then rm scripts directory for every command.

Pyinfra advantages:

• No script sync whatsoever, just change the file and run directly. Each line is a seperate ssh connection.
• Can upload files mid-script because "Each line is a seperate ssh connection."
• No need to enter sudo for your password every time. Passed via env var, used with SUDO_ASKPASS
• No fiddling with single or double quotes. Python allows you to use triple quotes.
• Syntax check (logic flow only, no shell) happens before you run the script.

Pyinfra disadvantages:

• Pyinfra operations need to be written seperately for apt, rpm, windows, etc. anyway. There's no unification.
• Shell syntax highlighting is crap because it is in strings.
• You're writing shell anyway, wrapped in python.
• sudo in uploaded .sh scripts does not work because pyinfra cannot prompt for sudo inside of a script, only for the parent command itself. Pretty much delegates .sh to local only.
• 🙄 Pyinfra is still over-engineered af, but you can simplify things by combining group_data into the hosts files, and using the scripts/ + servers/ pattern and aliasing stuff like server.shell() to run()
• Output tends to be hidden unless you use -vvv or another

Pyinfra Notes:

• 🙄 Use on localhost with @local

Notes for other systems.

• https://stackoverflow.com/questions/3586106/perform-commands-over-ssh-with-python
• Keep it simple. No libraries required:

import subprocess
subprocess.Popen(f"ssh {user}@{host} {cmd}", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()

• Issue is when you need to use sudo password.
• Spur: https://github.com/mwilliamson/spur.py
  • Nice but does not create a shell (no access to variables etc. without using bash -c "...")
• Pyinvoke: Just shell wrapped in python calls. It's pyinfra without the SSH.
• Fabric:
  • https://github.com/yoongkang/fabric-deployment
  • https://yoongkang.com/blog/fabric-deployment/

Nice one-off commands in pure Shell + SSH

• Rsync: rsync -axvz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p64" --progress ./README.md [email protected]:/home/gnat/Desktop/stuff/
• SystemD: ssh -i ../keys/changeme/changeme [email protected] 'systemctl --user stop Desktop'
• Run script: ssh -i ../keys/changeme/changeme [email protected] < script.sh

Rough equivalent of Pyinfra in pure shell.

• Another implementation: https://github.com/cbowdon/Bashistrano

#!/bin/bash
set -euo pipefail

SERVER=$1
SCRIPT=$2

# Error, display help.
if [[ -z "$SERVER" || -z "$SCRIPT" ]]; then echo "[$0] 🛟 Usage: $0 ./servers/server.sh ./scripts/script.sh"; exit; fi

. $SERVER

echo "[$0] 🌐 Running $2 ➡️  $IP ($1)"

SCRIPT_FILE=$(basename $SCRIPT)
SCRIPT_DIR=$(dirname $SCRIPT)

echo "[$0] 🔌 ssh $SSH_OPTS $USER@$IP"

#ssh_check_fallback_port_22

# Uploading the script each time means you can do updates, and just run it. 😎
# We upload the directory containing the script to support includes.
echo "[$0] 🚚 Uploading $SCRIPT_DIR ➡️  $USER@$IP:$WORKSPACE/scripts/"
rsync $RSYNC_OPTS "$RSYNC_SSH" "$SCRIPT_DIR/" $USER@$IP:$WORKSPACE/scripts/

echo "[$IP] 📡 Start =========================="
echo ""
# --login required for docker.
ssh $SSH_OPTS $USER@$IP "$SSH_ENV bash --login -c '$WORKSPACE/scripts/$SCRIPT_FILE'"
echo ""
echo "[$IP] 📡 End ============================"
echo "[$0] 🧹 Cleaning $WORKSPACE/scripts/*"
echo "ssh $SSH_OPTS $USER@$IP \"'$SSH_ENV rm -rf $WORKSPACE/scripts/'\""

ssh $SSH_OPTS $USER@$IP "$SSH_ENV bash --login -c 'rm -rf $WORKSPACE/scripts/'"
echo "[$0] 🏁 Finished"

Rough equivalent of Pyinfra in shell. Early verion.

#!/bin/bash
set -euo pipefail

USERNAME="gnat"
PASSWORD="..."
SERVERS=("127.0.0.1")

copy() { rsync -axvz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress "$@" ${USERNAME}@${SERVERS[i]}:/home/${USERNAME}/Desktop/upload ;}

run_sudo() { ssh -o StrictHostKeyChecking=no -tt -i ../keys/changeme ${USERNAME}@${SERVERS[i]} sudo bash -c $@ ;}

run() {	ssh -o StrictHostKeyChecking=no -tt -i ../keys/changeme ${USERNAME}@${SERVERS[i]} bash -c $@ ;}

for i in ${!SERVERS[*]} ; do
	echo ${SERVERS[i]}
	copy ./README.md
	run_sudo "whoami"
	run "whoami"
	run_sudo "whoami; echo ${SERVERS[i]}"
	# Works, but entering your password each time is stupid.
done