stevehenderson

# First run this in top cell:  pip install pyhive
from pandas import read_sql
from pyhive import presto

presto_conn = presto.Connection(host="collab-cluster-hostname-m", schema="your_presto_schema", catalog="presto_catalog", port=8060, username ="presto")
presto_cursor = presto_conn.cursor()

# Query presto node
query = "select * FROM system.runtime.nodes"
query_result = read_sql(query, presto_conn) 

stevehenderson

curl ipinfo.io/ip
curl api.ipify.org
curl checkip.dyndns.org
dig +short myip.opendns.com @resolver1.opendns.com
host myip.opendns.com resolver1.opendns.com
curl ident.me
curl bot.whatismyipaddress.com
curl ipecho.net/plain
curl ifconfig.me
curl -4/-6 icanhazip.com

stevehenderson

- id: "Read Values"
  name: ubuntu
  entrypoint: bash
  args:
    - -c
    - |
      # Read from "/workspace"
      echo "Where am I " $(pwd)
      echo "Contents " $(ls)  

stevehenderson

# The following sets the over all timeout
timeout: 7200s
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://project-volumes/big-project/big-file.tar', '.']
  waitFor: ['-']  # The '-' indicates that this step begins immediately.
  timeout: 4600s  # Per step timeout

stevehenderson

Sometimes you delete stuff on GKE Autopilot and it doesn't take:

kubectl delete pv troubledvolume --grace-period=0 --force
kubectl patch pv troubledvolume -p '{"metadata": {"finalizers": null}}'

stevehenderson

So I'm playing around with Harmony @ cs.cornell.edu.

Install steps (Ubuntu 20) [Should work on WSL]

Install Prequisites

You need to have build-essential, clang, python3 + dev headers, and graphviz

I also always install anything python related in its own virtual environments, so I added the python3-venv package

stevehenderson

kubectl get pods --namespace somens -o wide | awk '{print $7,"   ", $1}' | sort

stevehenderson

How do you unmarshall nested json

Given dgraph res.Json:

"q": [
      {
        "city": {
 "uid": "0x51c7ebb",

stevehenderson

gcloud sql import csv cloud-sql-instance-name gs://somebucket/folder/2/bigolecsv.gz -d somedatabase --table sometable

stevehenderson

#!/bin/bash
#
# A simple entropy walker.  
# Requirements:  apt install ent
#
echo 0,File-bytes,Entropy,Chi-square,Mean,Monte-Carlo-Pi,Serial-Correlation
for i in *; 
do echo $i $(ent "$i" -t | tail -n1); 
done