sshkarupa

#Spider Websites with Wget – 20 Practical Examples

Wget is extremely powerful, but like with most other command line programs, the plethora of options it supports can be intimidating to new users. Thus what we have here are a collection of wget commands that you can use to accomplish common tasks from downloading single files to mirroring entire websites. It will help if you can read through the wget manual but for the busy souls, these commands are ready to execute.

1. Download a single file from the Internet
wget http://example.com/file.iso

2. Download a file but save it locally under a different name
wget ‐‐output-document=filename.html example.com

sshkarupa

#!/bin/bash
# Stop all containers
docker stop $(docker ps -a -q)
# Delete all containers
docker rm $(docker ps -a -q)
# Delete all images
docker rmi $(docker images -q)

sshkarupa

# (I recommend understanding the basics of this first: http://sequel.jeremyevans.net/rdoc/files/doc/object_model_rdoc.html)

# Extending the underlying dataset  (http://sequel.jeremyevans.net/rdoc/files/README_rdoc.html#label-Extending+the+underlying+dataset)

# The recommended way to implement table-wide logic by defining methods on the dataset using dataset_module:

    class Post < Sequel::Model
      dataset_module do
        def posts_with_few_comments
          where{num_comments < 30}

sshkarupa

# HOW TO INSTALL JANUS
curl -Lo- http://bit.ly/janus-bootstrap | bash

#errored out...had to install git first, derrr
sudo apt-get install git
curl -Lo- http://bit.ly/janus-bootstrap | bash

# error! had to install rake next
sudo apt-get install rake
curl -Lo- http://bit.ly/janus-bootstrap | bash

sshkarupa

How to install PhantomJS on Ubuntu

Version: 1.9.8

Platform: x86_64

First, install or update to the latest system software.

sudo apt-get update
sudo apt-get install build-essential chrpath libssl-dev libxft-dev

sshkarupa

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname

sshkarupa

#!/bin/bash

# Bash script to install latest version of ffmpeg and its dependencies on Ubuntu 12.04 or 14.04
# Inspired from https://gist.github.com/faleev/3435377

# Remove any existing packages:
sudo apt-get -y remove ffmpeg x264 libav-tools libvpx-dev libx264-dev

# Get the dependencies (Ubuntu Server or headless users):
sudo apt-get update

sshkarupa

## just some ways to check if a url exists

# method 1 - from Simone Carletti
require "net/http"
url = URI.parse("http://www.google.com/")
req = Net::HTTP.new(url.host, url.port)
res = req.request_head(url.path)

# method 2 - from some kid on the internet
require 'open-uri'

sshkarupa

# http://askubuntu.com/questions/505446/how-to-install-ubuntu-14-04-with-raid-1-using-desktop-installer
# http://askubuntu.com/questions/660023/how-to-install-ubuntu-14-04-64-bit-with-a-dual-boot-raid-1-partition-on-an-uefi%5D

sudo -s
apt-get -y install mdadm
apt-get -y install grub-efi-amd64
sgdisk -z /dev/sda
sgdisk -z /dev/sdb
sgdisk -n 1:0:+100M -t 1:ef00 -c 1:"EFI System" /dev/sda
sgdisk -n 2:0:+4G -t 2:fd00 -c 2:"Linux RAID" /dev/sda

sshkarupa

### Unicorn ###
check process unicorn 
  with pidfile "/home/deployer/qna/current/tmp/pids/unicorn.pid"
  start program = "/bin/su - deployer -c 'cd /home/deployer/qna/current && RAILS_ENV="production" /home/deployer/.rbenv/bin/rbenv exec bundle exec unicorn -c /home/deployer/qna/current/config/unicorn/production.rb -E deployment -D'"
  stop program = "/bin/su - deployer -c 'kill -0 `cat /home/deployer/qna/current/tmp/pids/unicorn.pid`'"
  if memory usage > 90% for 3 cycles then restart
  if cpu > 90% for 2 cycles then restart
  if 5 restarts within 5 cycles then timeout

### Nginx ###