jonico

Scripts to determine PostgreSQL database size and rows processed

Differences between PostgreSQL and MySQL storage format and why this matters for billing estimations

tl;dr Rows processed numbers between Postgres and MySQL and database size may differ due to different index and row storage

PostgreSQL and MySQL are both relational databases with strong transactional capabilities. The way their storage engines store rows and corresponding indexes and how those indexes are used during queries differs significantly though. Check out this article from Uber Engineering for the technical details behind those differences.

Due to those index and row storage format differences, any numbers about rows read / written and database size from PostgreSQL will differ from the numbers you can expect once migrated to MySQL. If you are using similar indexes for your queries, the numbers should be pretty similar but depending on your exact queries and read/write pa

jonico

MySQL on RDS configured for import to PlanetScale example

This folder contains an example Terraform configuration that deploys a MySQL database - using RDS in an Amazon Web Services (AWS) account - that is properly configured for PlanetScale's DB import feature.

It will make sure that the RDS database has the binlog exposed, gtid-mode set to ON and is using ROW-based replication. All these are pre-requisites for imports to work without zero downtime.

If you are going to write a lot of data in your database in a very short amount of time, don't forget the only manual step after the Terraform setup.

jonico

PLANETSCALE_DB=brandnewdb
PLANETSCALE_BRANCH=mybranch
PLANETSCALE_ORG=jonico
PLANETSCALE_SERVICE_TOKEN=pscale_tkn_loCzIH7NktDK-GWJ71eX97Qr5D3a9iEO_pgHCSHUtw  
PLANETSCALE_SERVICE_TOKEN_NAME=69xrlIwgs4ms

jonico

#!/bin/sh

# shell script that waits for a PlanetScale branch to be ready for use and increases retry times exponentially

#  usage: wait-for-ps-branch-readiness.sh <db name> <branch name> <max retries>

function wait_for_branch_readiness {
    local retries=$1
    local db=$2
    local branch=$3

jonico

# copy Azure bloc storage files to S3 buckets
#
# Usage:
#   ./copy-azure-blob-storage-to-s3.sh <blob storage container> <s3 bucket>
#
# Example:
#   ./copy-azure-blob-storage-to-s3.sh my-container s3://my-bucket
#
# Note:
#   This script requires a working Azure CLI.

jonico

#!/usr/bin/bash
cd
mkdir -p /tmp/actions
git clone https://github.com/github/platform-samples.git
sdk install groovy
cd platform-samples/api/groovy/
groovy ListReposInOrg.groovy -t $GH_PUBLIC actions > ~/repo-list

cd
wget https://github.com/actions/actions-sync/releases/download/v202009231612/gh_202009231612_linux_amd64.tar.gz 

jonico

apiVersion: v1
items:
- apiVersion: v1
  kind: Pod
  metadata:
    creationTimestamp: "2021-04-29T15:09:09Z"
    labels:
      pod-template-hash: 65f67dcb66
      runner-deployment-name: actions-runner-deployment-epona
      runner-template-hash: 6788f4694c

jonico

# Replace the string "Hyrule" with your search pattern - only run on backup system - will take a LOOOONG time to run through
ghe-console -y <<'ENDSCRIPT'
  Repository.find_each do |repository|
    if repository.active && !repository.empty? then
      host,path = GitHub::Routing.lookup_route(repository.nwo)
        puts "Processing repo #{repository.nwo} ..."
        refs=`git --git-dir=#{path} rev-list --branches --tags | sort | uniq`
        refs.each_line do |ref|
          if system("git --git-dir=#{path} grep 'Hyrule' #{ref}")
            puts "Found pattern in ref #{ref} in repo #{repository.nwo}: "

jonico

ghe-console -y <<'ENDSCRIPT'
ActiveRecord::Base.connected_to(role: :reading) do
  puts "Version 0.2.0"
  emails = Set.new
  start_time = 90.days.ago.beginning_of_day
  Repository.where(active: true).find_each do |repo|
    Push
      .where(repository: repo)
      .where("created_at >= ?", start_time)
      .find_each do |push|

jonico

name: Build and publish Docker image + Container Scanning
on: [push]
jobs:

  build:
    name: Build
    runs-on: ubuntu-latest
    outputs:
      dockerimage: ${{ steps.dockerimage.outputs.dockerimage }}
      enablecontainerscanning: ${{ steps.enablecontainerscanning.outputs.enablecontainerscanning }}