kevin moore kevmo

Web Scraping Workshop

Using Requests and Beautiful Soup, with the most recent Beautiful Soup 4 docs.

Getting Started

Install our tools (preferably in a new virtualenv):

pip install beautifulsoup4

tmux cheat sheet

(C-x means ctrl+x, M-x means alt+x)

Prefix key

The default prefix is C-b. If you (or your muscle memory) prefer C-a, you need to add this to ~/.tmux.conf:

remap prefix to Control + a

Collections

each(array)

Underscore example:

_.each([1, 2, 3], function(num) { alert(num); });

Deploying a subfolder to GitHub Pages

Sometimes you want to have a subdirectory on the master branch be the root directory of a repository’s gh-pages branch. This is useful for things like sites developed with Yeoman, or if you have a Jekyll site contained in the master branch alongside the rest of your code.

For the sake of this example, let’s pretend the subfolder containing your site is named dist.

Step 1

Remove the dist directory from the project’s .gitignore file (it’s ignored by default by Yeoman).

#Setting up Nginx on Your Local System ###by Keith Rosenberg

##Step 1 - Homebrew The first thing to do, if you're on a Mac, is to install homebrew from http://mxcl.github.io/homebrew/

The command to type into terminal to install homebrew is:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

#Error management in gulp

Sucking at something is the first step to becoming sorta good at something

No one can assure you, that plugins will run smooth in any circumstances (except for tests - they could), so neither should you convince anyone, that your plugin will never break. Only thing, that you could possibly do (if something gone wrong) - is gracefully inform your plugin user, that something went wrong and die.

We are will use this plugin from beginning to demonstrate error management. Suppose you have a task in gulpfile.js that contains this code (we modified it a little bit to be closer to real-usage):

var coffee = require('gulp-coffee');

	echo 'export PATH=$HOME/local/bin:$PATH' >> ~/.bashrc
	. ~/.bashrc
	mkdir ~/local
	mkdir ~/node-latest-install
	cd ~/node-latest-install
	curl http://nodejs.org/dist/node-latest.tar.gz \| tar xz --strip-components=1
	./configure --prefix=~/local
	make install # ok, fine, this step probably takes more than 30 seconds...
	curl https://www.npmjs.org/install.sh \| sh

	<!DOCTYPE html>
	<html>
	<head>
	<title>Box Shadow</title>

	<style>
	.box {
	height: 150px;
	width: 300px;
	margin: 20px;

	// Just before switching jobs:
	// Add one of these.
	// Preferably into the same commit where you do a large merge.
	//
	// This started as a tweet with a joke of "C++ pro-tip: #define private public",
	// and then it quickly escalated into more and more evil suggestions.
	// I've tried to capture interesting suggestions here.
	//
	// Contributors: @r2d2rigo, @joeldevahl, @msinilo, @_Humus_,
	// @YuriyODonnell, @rygorous, @cmuratori, @mike_acton, @grumpygiant,

	<!DOCTYPE html>

	<html lang="en">
	<head>
	<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
	<title>imgur oauth</title>
	<script src="http://code.jquery.com/jquery.min.js"></script>
	<script>
	$(function () {
	var extractToken = function(hash) {