Skip to content

Instantly share code, notes, and snippets.

View stevehenderson's full-sized avatar

Steve Henderson stevehenderson

View GitHub Profile
@stevehenderson
stevehenderson / gsutil_ls_recursive.sh
Created February 3, 2023 00:54
gsutil gcs ls recursive
gsutil ls gs://some-bucket/**
@stevehenderson
stevehenderson / entropy-walker.sh
Created February 2, 2023 04:28
Entropy Walker - get entropy of all files in a directory
#!/bin/bash
#
# A simple entropy walker.
# Requirements: apt install ent
#
echo 0,File-bytes,Entropy,Chi-square,Mean,Monte-Carlo-Pi,Serial-Correlation
for i in *;
do echo $i $(ent "$i" -t | tail -n1);
done
@stevehenderson
stevehenderson / cloud_sql_csv_import.sh
Created December 7, 2022 03:58
Cloud SQL command line CSV import
gcloud sql import csv cloud-sql-instance-name gs://somebucket/folder/2/bigolecsv.gz -d somedatabase --table sometable
@stevehenderson
stevehenderson / golang_define_struct.md
Created November 30, 2022 14:29
golang unmarshall nested json from result

How do you unmarshall nested json

Given dgraph res.Json:

"q": [
      {
        "city": {
 "uid": "0x51c7ebb",
@stevehenderson
stevehenderson / k8s_nodes_pods.sh
Created November 10, 2022 17:30
Kubernetes pods and their node (for namespace x)
kubectl get pods --namespace somens -o wide | awk '{print $7," ", $1}' | sort
@stevehenderson
stevehenderson / harmony_install.md
Last active September 11, 2022 20:25
Installing Harmony

So I'm playing around with Harmony @ cs.cornell.edu.

Install steps (Ubuntu 20) [Should work on WSL]

Install Prequisites

You need to have build-essential, clang, python3 + dev headers, and graphviz

I also always install anything python related in its own virtual environments, so I added the python3-venv package

@stevehenderson
stevehenderson / force_delete_k8s_pv.md
Last active August 27, 2022 13:58
Delete Kubernetes Persistent Volume

Sometimes you delete stuff on GKE Autopilot and it doesn't take:

kubectl delete pv troubledvolume --grace-period=0 --force
kubectl patch pv troubledvolume -p '{"metadata": {"finalizers": null}}'
@stevehenderson
stevehenderson / cloud_build_large_file_copy_gcs.yaml
Last active August 13, 2022 19:32
Large File Timeout for Cloud Build gsutil
# The following sets the over all timeout
timeout: 7200s
- name: 'gcr.io/cloud-builders/gsutil'
args: ['cp', 'gs://project-volumes/big-project/big-file.tar', '.']
waitFor: ['-'] # The '-' indicates that this step begins immediately.
timeout: 4600s # Per step timeout
- id: "Read Values"
name: ubuntu
entrypoint: bash
args:
- -c
- |
# Read from "/workspace"
echo "Where am I " $(pwd)
echo "Contents " $(ls)
@stevehenderson
stevehenderson / ip_curl.sh
Created August 13, 2022 01:48
Find IP with curl
curl ipinfo.io/ip
curl api.ipify.org
curl checkip.dyndns.org
dig +short myip.opendns.com @resolver1.opendns.com
host myip.opendns.com resolver1.opendns.com
curl ident.me
curl bot.whatismyipaddress.com
curl ipecho.net/plain
curl ifconfig.me
curl -4/-6 icanhazip.com