Volker Hilsenstein VolkerH

How to Setup Automatic Uploads to Anaconda from Travis CI in 15 minutes

TL;DR: Edit .travis.yaml to install Anaconda and to run conda_upload.sh after testing. Edit meta.yaml to take in the environmental variables $VERSION and $CONDA_BLD_PATH. Create conda_upload.sh which sets the needed environmental variables, builds the tar archive, and uploads it to Anaconda. Finally edit some stuff on your Anaconda and Travis CI account so they can talk.

Intro

The following steps will detail how to automatically trigger Anaconda builds and uploads from Travis CI. This will only upload successful builds in the master branch and if there are multiple commits in a single day, it'll only keep the latest one. Both of these settings can easily be changed.

Edit .travis.yaml

First, edit .travis.yml so that it installs Anaconda.

install:

https://github.com/aancel/admin/wiki/VirtualGL-on-Ubuntu

https://virtualgl.org/About/Introduction

https://www.turbovnc.org

VirtualGL

When you use ssh with X forwarding, you might have noticed that you cannot execute programs that require 3D acceleration. That's where VirtualGL comes into play.

SLURM param sweeper

Get the code here.

This python library generates SLURM submission scripts which launch multiple jobs to 'sweep' a given set of parameters; that is, a job is run for every possible configuration of the params.

For example, you might have a script which accepts parameters a, b and c, which you call (passing a=1, b=2, c=3) like

	import paramiko
	k = paramiko.RSAKey.from_private_key_file("/Users/whatever/Downloads/mykey.pem")
	c = paramiko.SSHClient()
	c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
	print "connecting"
	c.connect( hostname = "www.acme.com", username = "ubuntu", pkey = k )
	print "connected"
	commands = [ "/home/ubuntu/firstscript.sh", "/home/ubuntu/secondscript.sh" ]
	for command in commands:
	print "Executing {}".format( command )


	import time
	import numpy as NP
	from redis import StrictRedis as redis

	# a 2D array to serialize
	A = 10 * NP.random.randn(10000).reshape(1000, 10)

	# flatten the 2D NumPy array and save it as a binary string
	array_dtype = str(A.dtype)

	"""Generate zone plate to scale for lithography manufacturing.

	Example usage:

	>>> # Create a 140 mm focal length zone plate for 625nm light where
	>>> # the lithographer is accurate to 10160 dpi.

	>>> from zone_plate import NormalFZP

	>>> zp = NormalFZP(f=140, w=625, thinnest_zone=25.4/10160)

	#
	# Hi all,
	# this is the Python code I used to make the visualization "Temperature circle"
	# (https://twitter.com/anttilip/status/892318734244884480).
	# Please be aware that originally I wrote this for my tests only so the
	# code was not ment to be published and is a mess and has no comments.
	# Feel free to improve, modify, do whatever you want with it. If you decide
	# to use the code, make an improved version of it, or it is useful for you
	# in some another way I would be happy to know about it. You can contact me
	# for example in Twitter (@anttilip). Unchecked demo data (no quarantees)

	from PIL import Image, ImageDraw, ImageFont
	from colour import Color
	import numpy as np

	# The function convert an image to ascii art
	# f: Input filename
	# SC: the horizontal pixel sampling rate. It should be between 0(exclusive) and 1(inclusive). The larger the number, the more details in the output.
	# If you want the ascii art output be the same size as input, use ~ 1/ font size width.
	# GCF: >0. It's an image tuning factor. If GCF>1, the image will look brighter; if 0<GCF<1, the image will look darker.
	# out_f: output filename

	#
	# Hi all,
	# this is the Python code I used to make the visualization
	# "Temperature anomalies, zonal means, 1900 - 2016"
	# (https://twitter.com/anttilip/status/921809347658895361).
	# Please be aware that originally I wrote this just to see how this type
	# of visualization would work. The code was not ment to be published
	# and therefore has only a couple of/no comments and is most likely
	# not written in the most optimal way.
	#