Andrew Sears asears

Requirements

no upfront installation/agents on remote/slave machines - ssh should be enough
application components should use third-party software, e.g. HDFS, Spark's cluster, deployed separately
configuration templating
environment requires/asserts, i.e. we need a JVM in a given version before doing deployment
deployment process run from Jenkins

Solution

If anyone is interested in setting up their system to automatically (or manually) sign their git commits with their GPG key, here are the steps:

Generate and add your key to GitHub
$ git config --global commit.gpgsign true ([OPTIONAL] every commit will now be signed)
$ git config --global user.signingkey ABCDEF01 (where ABCDEF01 is the fingerprint of the key to use)
$ git config --global alias.logs "log --show-signature" (now available as $ git logs)
$ git config --global alias.cis "commit -S" (optional if global signing is false)
$ echo "Some content" >> example.txt
$ git add example.txt
$ git cis -m "This commit is signed by a GPG key." (regular commit will work if global signing is enabled)

Using py.test is great and the support for test fixtures is pretty awesome. However, in order to share your fixtures across your entire module, py.test suggests you define all your fixtures within one single conftest.py file. This is impractical if you have a large quantity of fixtures -- for better organization and readibility, you would much rather define your fixtures across multiple, well-named files. But how do you do that? ...No one on the internet seemed to know.

Turns out, however, you can define fixtures in individual files like this:

tests/fixtures/add.py

import pytest

@pytest.fixture

	<!DOCTYPE html>
	<html>
	<head>
	<meta charset=utf-8>
	<title>Internet Archive MS-DOS Games</title>
	<style>li { list-style-type: none; margin-bottom: 1em }</style>
	</head>
	<body>
	<ul>
	<li><a href="https://archive.org/details/msdos_Nam_1965-1975_1991">'Nam 1965-1975 (1991)</a><br>

	### SYMBOLIC LINK FILES
	# Create a new symbolic link file named MySymLinkFile.txt in C:\Temp which links to $pshome\profile.ps1
	cd C:\Temp
	New-Item -ItemType SymbolicLink -Name MySymLinkFile.txt -Target $pshome\profile.ps1 # File
	# Target is an alias to the Value parameter

	# Equivalent to above
	New-Item -ItemType SymbolicLink -Path C:\Temp -Name MySymLinkFile.txt -Value $pshome\profile.ps1

	# Equivalent to above

	USER='admin'
	PASS='admin'
	CLUSTER='dev'
	HOST=$(hostname -f):8080

	function start(){
	curl -u $USER:$PASS -i -H 'X-Requested-By: ambari' -X PUT -d \
	'{"RequestInfo": {"context" :"Start '"$1"' via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' \
	http://$HOST/api/v1/clusters/$CLUSTER/services/$1
	}

	### An example of building a TensorFlow model from R using rPython ###
	# For this script you need to
	# 1. Have python 2.7 installed.
	# 2. Install the rPython package in R.
	# 3. Install Google's TensorFlow library as per these instructions:
	# http://www.tensorflow.org/get_started/os_setup.md#binary_installation

	### Here is how to setup and run a trivial TensorFlow model ###

	# Load TensorFlow (I couldn't get this to work without setting sys.argv... )

	import requests
	import json
	import sys
	import csv


	def getIssues(url):
	"""
	@param {url} The url of the github issues
	@return the json object

	FROM r-base:latest
	COPY . /usr/local/src/myscripts
	WORKDIR /usr/local/src/myscripts
	CMD ["Rscript", "myscript.R"]

	#!/usr/bin/env bash

	# When will my Lenovo order arrive?
	#
	# I grew impatient while waiting for my Thinkpad to ship, and the arrival date
	# kept changing, so I wrote this script to scrape their order details page.
	#
	# Might not work on all platforms, and it's parsing HTML with sed, so there be
	# plenty of dragons within this script.