This is pretty much what I had intended to work on. Specifically I'd like to get the docker images that package up spark in the ipython/scipyserver docker image (https://github.com/rdhyee/ipython-spark/blob/master/Dockerfile) to run on a Mesos cluster. Of relevance is a Spark PR that seems almost ready to go: apache/spark#3074 (combined with https://issues.apache.org/jira/browse/SPARK-2691). If we get the basics working, I'd love to work such integration with https://github.com/rgbkrk/cloudpipe, which I understand to be a hybrid of http://www.multyvac.com/ (I loved its predecessor: picloud) + tmpnb (https://lambdaops.com/ipythonjupyter-tmpnb-debuts/). Wonderful to be able to let people spin up a temporary Jupyter notebook that can run Spark and also be based on an arbitrary docker image.
I hereby claim:
- I am rdhyee on github.
- I am rdhyee (https://keybase.io/rdhyee) on keybase.
- I have a public key whose fingerprint is 78E8 E8E8 B007 206C 2A5D 9C83 AA61 D643 ECB4 CD2D
To claim this, I am signing this object:
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Docker Upstart and SysVinit configuration file | |
# Customize location of Docker binary (especially for development testing). | |
#DOCKER="/usr/local/bin/docker" | |
# Use DOCKER_OPTS to modify the daemon startup options. | |
#DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4" | |
# If you need Docker to use an HTTP proxy, it can also be specified here. | |
#export http_proxy="http://127.0.0.1:3128/" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ubuntu@ip-172-30-2-96:~/repro-demo$ sudo docker images | |
sudo: unable to resolve host ip-172-30-2-96 | |
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE | |
repro/dexy latest 21d9af2ecf61 2 minutes ago 1.274 GB | |
ubuntu 14.04 ba5877dc9bec 7 days ago 192.7 MB | |
busybox latest a9eb17255234 7 weeks ago 2.433 MB |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def wh_geocode(): | |
import requests | |
url = "http://rpc.geocoder.us/service/csv?address=1600+Pennsylvania+Ave,+Washington+DC" | |
r = requests.get(url).content | |
return r.split(",") |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0" encoding="utf-8"?> | |
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en-us"><title>Latest wishbooks for rdhyee on unglue.it</title><link href="http://localhost:8000/supporter/rdhyee/feed/" rel="alternate"></link><link href="http://localhost:8000/supporter/rdhyee/feed/" rel="self"></link><id>http://localhost:8000/supporter/rdhyee/feed/</id><updated>2014-06-19T14:25:39Z</updated><entry><title>23rd Century Romance</title><link href="http://localhost:8000/work/130413" rel="alternate"></link><author><name>Jake Hartnell</name></author><id>http://localhost:8000/work/130413</id><summary type="html">23rd Century Romance</summary></entry><entry><title>Reinventing Bach</title><link href="http://localhost:8000/work/113764" rel="alternate"></link><author><name>Paul Elie</name></author><id>http://localhost:8000/work/113764</id><summary type="html">Reinventing Bach</summary></entry><entry><title>A Lector's Guide and Commentary to the Revised Common Lectionary, Year A</title><link href="http://localhost |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<feed xmlns:dcterms="http://purl.org/dc/terms/" xmlns:opds="http://opds-spec.org/" xmlns="http://www.w3.org/2005/Atom" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.kbcafe.com/rss/atom.xsd.xml" xsi:schemaLocation="http://purl.org/dc/elements/1.1/ http://dublincore.org/schemas/xmls/qdc/2008/02/11/dc.xsd http://purl.org/dc/terms/ http://dublincore.org/schemas/xmls/qdc/2008/02/11/dcterms.xsd"> | |
<title>Unglue.it Catalog: Creative Commons Books</title> | |
<id>http://localhost:8000/opds/creative_commons</id> | |
<updated>2014-07-11T21:04:18.442057+00:00</updated> | |
<author> | |
<name>unglue.it</name> | |
<uri>http://localhost:8000</uri> | |
</author> | |
<link href="https://unglue.it/opds" type="application/atom+xml;profile=opds-catalog;kind=navigation" rel="start"/> | |
<entry> |