Skip to content

Instantly share code, notes, and snippets.

View dkam's full-sized avatar

Dan Milne dkam

View GitHub Profile
@dkam
dkam / gist:37d7a41d3c4ab00278580531832130bd
Last active November 27, 2023 02:07
Postgres table, index & toast size.
# This ( https://gist.github.com/hatarist/675dc3debf6cf5f825b5c15aed4cbac0 ) with TOAST size formatting
SELECT
table_name,
pg_size_pretty(table_bytes) AS table,
pg_size_pretty(index_bytes) AS index,
pg_size_pretty(toast_bytes) AS toast,
pg_size_pretty(total_bytes) AS total
FROM (
SELECT
@dkam
dkam / songlink.rb
Created October 31, 2023 02:41
Lookup albums on song.link for sharing
@dkam
dkam / add_query_parameters_to_a_url.rb
Created September 15, 2023 05:15
Add query parameter with URI
URI(url).tap {|u| u.query = URI.encode_www_form(CGI.parse(u.query || "").merge({ref: 123})) }
@dkam
dkam / misc.rb
Created September 5, 2023 06:08
Logging in rails
# Make add tags to the normal rails logger, accessed like `hc_logger.info("my log message")`
def hc_logger = @hc_logger ||= logger.tagged("HttpClients").tagged(self.class.name)
# Create a new logger as an instance variable for a class:
class MyServiceClass
# ....
def my_logger = @my_logger ||= create_logger.tagged(self.class.name)
@dkam
dkam / bot_shedding.rb
Last active August 10, 2023 02:06
A Rails before_action to request bots retry-after X seconds when normalised load is high
def bot_shedding(wait: 600, threshold: 1)
require "sys/cpu"
require "etc"
# normalize the 1-minute load average based on processor count
return unless Sys::CPU.load_avg[0] / Etc.nprocessors > threshold
return unless DeviceDetector.new(request.user_agent).bot?
logger.info("Shedding bots: Returning 503 / Retry-After: #{wait} for #{request.remote_ip} / #{request.user_agent}")
/\b(?:97[89]-?)?(?:\d{1,5}-?)?(?:\d{1,7}-?)?(?:\d{1,6}-?)?\d{1,3}\b/
@dkam
dkam / Moviehash.rb
Last active October 27, 2024 11:40
Open Subtitles Hash in Ruby
# This version works with files and urls. For URLs, it only downloads the chunks
# of the file neccessary to calculate the hash.
#
# Author: Dan Milne
require "net/http"
require "uri"
module Moviehash
class Error < StandardError; end
@dkam
dkam / free.rb
Created May 19, 2023 04:10
Get free disk space
#!/usr/bin/env ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'json'
gem 'sys-filesystem'
end
@dkam
dkam / valid_json.rb
Last active April 23, 2023 07:43
Check JSON is valid
def valid_json?(string)
result = JSON.parse(string)
result.is_a?(Hash) || result.is_a?(Array)
rescue JSON::ParserError, TypeError
false
end
def valid_json(string, default: [])
result = JSON.parse(string)
@dkam
dkam / curlr.rb
Last active April 24, 2023 06:44
Basic http client which can process html and xml
#!/usr/bin/env ruby
require 'open-uri'
require 'nokogiri'
VERSION = 0.1
def getter(url)
URI.open(url, "User-Agent" => "Curlr/#{VERSION}")
rescue OpenURI::HTTPError => e
e.io