Skip to content

Instantly share code, notes, and snippets.

@mahemoff
mahemoff / parallel_array.rb
Created June 29, 2019 08:57
Extending Array with Parallel gem
class Array
%w(each map each_with_index map_with_index flat_map any? all?).each { |meth|
define_method("#{meth}_in_parallel") { |&block|
Parallel.send(meth, self) { |item| block.call(item) }
}
}
}
@mahemoff
mahemoff / backup.sh
Created February 2, 2019 17:19
mysql incremental backup
#!/bin/bash
function slice() {
lower=$1
upper=$(expr $lower + 100000)
echo "Backing up $lower"
mysqldump db_name table_name --opt --no-create-info --where "id > $lower and id <= $upper" | gzip -c | ssh user@host "cat > /home/backup/dump$lower.gz"
}
lower=0
@mahemoff
mahemoff / docker-1604.error.md
Created January 9, 2019 00:37
Fixing Docker error on Ubuntu 16.04

As here, but refined ...

Create or edit /etc/systemd/system/containerd.service.d/override.conf to contain just this:

[Service] ExecStartPre=

To make it happen:

  • systemctl daemon-reload
@mahemoff
mahemoff / Ansible Disk Check
Last active October 16, 2024 23:08
Show disk space and warn about disk full in Ansible
* Shows a message while asserting like:
ok: [host] => {
"msg": "disk usage 4.2B of total 20.0GB (21.0%) (should be within limit 90.0%)"
}
* Note this only looks at first mount point on current node
* Fails if disk is near-full
* Last step pushes to a push-based monitoring service, which will alert us if it doesn't get there after some time
* Need to setup a variable `disk_limit`, which is the max acceptable usage ratio, e.g. set it to 0.8 if you want to keep disks within 80% of max size
@mahemoff
mahemoff / gist:07c0c3427aa3eb3c0abc07f76fc68279
Created September 15, 2018 08:24
Show disk space and warn about disk full in Ansible
- name: show disk space
debug: msg="{{ ((mount.size_total - mount.size_available) / 1000000000) | round(1,'common') }}GB of {{ (mount.size_total / 1000000000)|round(
1, 'common') }}GB ({{ (100 * ( (mount.size_total - mount.size_available) / mount.size_available)) | round(1, 'common')}}%)"
vars:
mount: "{{ ansible_mounts | first }}"
tags: disk
- name: e
@mahemoff
mahemoff / README.md
Last active August 6, 2018 09:58
Verifying Google OAuth auth code on the back-end

This is the "missing Ruby example" for the ID flow example at https://developers.google.com/identity/sign-in/web/server-side-flow.

It's easy enough to get an auth code like "4/BlahBlahBlah...", but I couldn't find any working examples on how to exchange it for the access code and encoded ID.

To use this, you need to access Google's API console, and under "credentials" establish a client ID and secret, which should go in your environment. (Most examples will use the "secrets.json" file, but I don't want to keep a separate config file for every platform, so it's better to put them in something like Rails' secret.yml or Figaro).

The auth_code is obtained from your web or native client using Google's front-end libraries. The client posts it to your own back-end, which does the exchange and verifies+stores the result. Note the redirect URI must be configured in Google's "credentials" console, otherwise the call will fail (even though it serves no purpose in this context; it's only needed for a non-JavaScript

@mahemoff
mahemoff / linodecost.rb
Created June 20, 2018 08:50
Report monthly Linode cost
#!/usr/bin/env ruby
require 'byebug'
require 'linode'
require 'awesome_print'
# setup API access - key is in file
open('/etc/linode.conf').read =~ /api_key: (.+)/
key=$1
lin = Linode.new(api_key: key).linode

MySQL backup commands and sizes

Adventures in storing and backing up a typical database using innobackupex and gpg. Using gzipped tar due to ubiquity, even though it's possibly 10-20% worse on perf/storage.

Sizes:

PREPARED (ie ready to move to MySQL folder)

  • 2800MB Uncompressed and prepared (roughly the size of live database)
  • 21000MB Uncompressed before compression
  • 700MB Crypted+Compressed (ideal for external storage)
@mahemoff
mahemoff / bench.rb
Last active March 11, 2018 08:36
Benchmarking performance of persistent HTTP requests
#!/usr/bin/env ruby
require 'uri'
require 'net/http'
require 'benchmark'
# dummy fetch first URL to baseline setup (ensures DNS and any routing
# optimisations done)
def prime_fetching(urls)
ignored = Net::HTTP.get_response(URI(urls.first))
end
@mahemoff
mahemoff / MySQL monitoring
Last active January 28, 2018 13:56
MySQL one-liner for monitoring long queries on the console
+---------+--------+------------+-------------------+---------+------+----------+------------------------------------------------------------------------------------------------------------------------------+
| ID | USER | HOST | DB | COMMAND | TIME | STATE | QUERY |
+---------+--------+------------+-------------------+---------+------+----------+------------------------------------------------------------------------------------------------------------------------------+
| 2786271 | appy | borg:89700 | global_app_center | Query | 12 | updating | SELECT `posts` FROM `blog` WHERE `authors`.`id` IN ( 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2.. | |
+---------+--------+------------+-------------------+---------+------+----------+---------------------------------------------------------------------