Skip to content

Instantly share code, notes, and snippets.

View robhammond's full-sized avatar

Rob Hammond robhammond

38 followers · 26 following
View GitHub Profile
@robhammond
robhammond / perl-dep-scanner.pl
Created July 24, 2012 08:52
Perl dependency scanner
#!/usr/bin/env perl
###
# This snippet will recursively scan directories for use/require statements
# in perl scripts or modules, making it easy to build a list of dependencies
# from code you've inherited or neglected to document.
###
use Perl::PrereqScanner;
use File::Spec::Functions qw( catfile );
use File::Find qw(finddepth);
use Data::Dumper;
@robhammond
robhammond / gist:4155823
Created November 27, 2012 17:46
Basic Mojo UserAgent parallel crawler
#!/usr/bin/env perl
use Modern::Perl;
use Mojo::UserAgent;
use Mojo::IOLoop;
use Mojo::URL;
# FIFO queue
my $seed_url = 'http://www.google.co.uk/';
my @urls = ($seed_url);
@robhammond
robhammond / url-analysis.pl
Created January 24, 2013 17:30
Analysis of keywords in different parts of URLs. You'll need to extract the URLs from Google's SERPs yourself and paste into the @urls array.
#!/usr/bin/env perl
use strict;
use Modern::Perl;
use Domain::PublicSuffix;
use URI;
my $keyword = 'broadband';
my @urls = qw(http://www.moneysupermarket.com/broadband/
http://www.o2.co.uk/broadband
@robhammond
robhammond / mojo-crawler
Last active January 16, 2018 09:17
Event source mojolicious crawler
package WebCrawl::Crawl;
use Mojo::Base 'Mojolicious::Controller';
sub crawl {
my $self = shift;
# Increase inactivity timeout for connection a bit
Mojo::IOLoop->stream($self->tx->connection)->timeout(15);
# Change content type
@robhammond
robhammond / search-metrics-charts.pl
Last active June 26, 2018 10:58
Get date and chart x-axis values from the JSON created when generating a SEO visibility chart in SearchMetrics. Makes it easier to plot YoY growth etc.
#!/usr/bin/env perl
use strict;
use Modern::Perl;
use Mojo::JSON qw(decode_json);
# seo-paid-visibility
my $raw = <<EOF;
{"data":{...}}
EOF
@robhammond
robhammond / web-servers.md
Last active August 29, 2015 13:57 — forked from willurd/web-servers.md

Each of these commands will run an ad hoc http static server in your current (or specified) directory, available at http://localhost:8000. Use this power wisely.

Discussion on reddit.

Python 2.x

$ python -m SimpleHTTPServer 8000
@robhammond
robhammond / reddit.pm
Created June 4, 2014 08:08
package Apis::Reddit;
use strict;
use warnings;
use Mojo::Log;
use DateTime;
use Mojo::UserAgent;
my $log = Mojo::Log->new();
sub new {
@robhammond
robhammond / 0_reuse_code.js
Created July 30, 2014 16:42
Here are some things you can do with Gists in GistBox.
// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console
@robhammond
robhammond / http-status-checker-async.pl
Last active October 1, 2015 21:29
Non-blocking HTTP status checker script using Mojolicious. Accepts text file 'urls.txt' and writes to 'http-status.csv'
#!/usr/bin/env perl
use strict;
use utf8;
use warnings qw(all);
use Modern::Perl;
use Mojo::Util qw(decode encode html_unescape xml_escape);
use Mojo::DOM;
use Mojo::Log;
use Mojo::Asset::File;
@robhammond
robhammond / nginx-toadfarm.conf
Created December 16, 2016 11:58
upstream myapp {
server 127.0.0.1:8082;
}
server {
listen 80;
server_name example.com;
location / {
proxy_set_header X-Request-Base https://$host/myapp;