Skip to content

Instantly share code, notes, and snippets.

View wenming's full-sized avatar

Wenming Ye wenming

  • Amazon Web Services
  • Redmond, WA
View GitHub Profile
@ellisonbg
ellisonbg / Configuring the IPython.parallel
Created August 20, 2011 18:11
Running IPython.parallel on Microsoft Azure
============================
Configuring IPython.parallel
============================
This guide describes the steps needed to run IPython engines on Microsoft Azure to perform a parallel
computation in the Microsoft cloud. We assume (see above guide) that Python, IPython and PyZMQ have
been installed on a set of Azure compute nodes.
1. Install Python, IPython and PyZMQ on a system having open access to the internet. We used an
Ubuntu 11.04 system for this purpose. This system will run the IPython controller and the IPython
@wenming
wenming / millionblobdownload.txt
Created August 13, 2012 01:24
downloading millions of files from blob storage fast
Raw notes for downloading 6.6+ million files from blob storage within hours using a few simple tools on a single machine.
1. Get a list of files from blob storage. A few lines of c# code will do.
//In app config.
<configuration>
<appSettings>
<add key="StorageConnectionString"
value="DefaultEndpointsProtocol=https;AccountName=storagename;AccountKey=yourkey" />
</appSettings>