Skip to content

Instantly share code, notes, and snippets.

@chilts
chilts / alexa.js
Created October 30, 2013 09:27
Getting the Alexa top 1 million sites directly from the server, unzipping it, parsing the csv and getting each line as an array.
var request = require('request');
var unzip = require('unzip');
var csv2 = require('csv2');
request.get('http://s3.amazonaws.com/alexa-static/top-1m.csv.zip')
.pipe(unzip.Parse())
.on('entry', function (entry) {
entry.pipe(csv2()).on('data', console.log);
})
;
@luckyshot
luckyshot / response.php
Last active November 1, 2024 08:20
Web scraping done right (with cUrl and user agent)
<?php return array (
'url' => 'https://xaviesteve.com/',
'content' => '<!doctype html><html>...</html>',
'cookies' => '__cfduid=d3fa669e1069e72c2e47d127ab9b8e11f1465390629',
'http_code' => 200,
'content_type' => 'text/html; charset=UTF-8',
'header_size' => 578,
'request_size' => 229,
'filetime' => -1,
'ssl_verify_result' => 0,
@brandonmwest
brandonmwest / gist:4512396
Last active June 25, 2022 13:11
curl + PHP example of using the SendGrid web api

This example uses cURL with PHP through our web API to send an email.

<?php $url = 'http://sendgrid.com/';
$user = 'USERNAME';
$pass = 'PASSWORD'; 
 
$params = array(
    'api_user'  => $user,
 'api_key' =&gt; $pass,
@maccath
maccath / gist:3981205
Created October 30, 2012 16:09
Split PDF to individual pages using FPDI and FPDF
<?php
/**
* Split PDF file
*
* <p>Split all of the pages from a larger PDF files into
* single-page PDF files.</p>
*
* @package FPDF required http://www.fpdf.org/
* @package FPDI required http://www.setasign.de/products/pdf-php-solutions/fpdi/
@evandrix
evandrix / README.md
Created September 11, 2012 00:06
Headless web browsers

Here are a list of headless browsers that I know about:

  • [HtmlUnit][1] - Java. Custom browser engine. JavaScript support/DOM emulated. Open source.
  • [Ghost][2] - Python only. WebKit-based. Full JavaScript support. Open source.
  • [Twill][3] - Python/command line. Custom browser engine. No JavaScript. Open source.
  • [PhantomJS][4] - Command line/all platforms. WebKit-based. Full JavaScript support. Open source.
  • [Awesomium][5] - C++/.Net/all platforms. Chromium-based. Full JavaScript support. Commercial/free.
  • [SimpleBrowser][6] - .Net 4/C#. Custom browser engine. No JavaScript support. Open source.
  • [ZombieJS][7] - Node.js. Custom browser engine. JavaScript support/emulated DOM. Open source.
  • [EnvJS][8] - JavaScript via Java/Rhino. Custom browser engine. JavaScript support/emulated DOM. Open source.