Gerald Rich newsroomdev

Using IBM Watson Speech to Text API to translate a ProPublica podcast

An example of using the Watson Speech to Text API to translate a podcast from ProPublica: How a Reporter Pierced the Hype Behind Theranos

This is just a simpler demo of the same technique I demonstrate to make automated video supercuts in this repo: https://github.com/dannguyen/watson-word-watcher

The transcription takes just a few minutes (less if you parallelize the requests to IBM) and is free...but it isn't perfect by any means. It doesn't fare super well on proper nouns:

Charles Ornstein's last name is transcribed as Orenstein
John Carreyrou's last name becomes John Kerry Roo

Demonstrating jenks natural breaks implemented in simple-statistics. "Ckmeans clustering is an improvement on heuristic-based clustering approaches like Jenks." It is "a dynamic programming approach to the problem of clustering numeric data into groups with the least within-group sum-of-squared-deviations"

Rendered by d3js, based on an example by Mike Bostock and Tom MacWright's original comparison with quantize.

More on ckmeans is in the Simple-Statistics documentation. Also see the PR removing Jenks here and the original narrative on how Jenks algorithm was reimplemented through Tom's literature review.

Here's our template for what should go in a pull request. This seems overly-verbose, but will help us:

manage technical debt
link pull requests to sprint tasks
give future developers a lot more context
give reviewers the context they need to understand the problem you're trying to solve and the acceptance criteria you're trying to meet.

It's nice, but not required to fill everything out when you first open your pull request. This template shouldn't be a barrier to contributing code - and you can open a PR with the words WIP in front, to let us know it's not quite there yet, then go back and update information at any time.

	library(tidyverse)
	library(sf)
	library(tigris)

	# start by picking a state from https://github.com/Microsoft/USBuildingFootprints
	# WARNING: these files can be pretty big. using arizona for its copious subdivisions and reasoanable 83MB.
	url_footprint <- "https://usbuildingdata.blob.core.windows.net/usbuildings-v1-1/Arizona.zip"
	download.file(url_footprint, "Arizona.zip")
	unzip("Arizona.zip")

	import asyncio
	import aiohttp
	import os
	import random
	import re
	import sys
	import traceback
	from io import StringIO
	from lxml.html import parse, make_links_absolute
	from lxml.cssselect import CSSSelector

	<html>
	<head>
	<!-- Load the d3 library. -->
	<script src="http://d3js.org/d3.v3.min.js" charset="utf-8"></script>
	<link href='http://fonts.googleapis.com/css?family=Open+Sans' rel='stylesheet' type='text/css'>
	<style>
	body { font-family: "Open Sans"; }
	text.stateID { dominant-baseline: middle; text-anchor: middle; }
	</style>
	</head>

	#~/.gnupg/gpg.conf
	# Hide your PGP version & other PGP version metadata with these two config lines.

	# This line hides the "Version: GnuPG vX.XX.XX" line
	no-emit-version

	# This line hides the "Comment: XXXXX" lines
	no-comments
	# NOTE if you are using Thunderbird+Enigmail you probably need to set this again in
	# the Enigmail->Preference settings:

	Next year, try to:

	* Drink more water.
	* Drink less coffee, beer and whiskey.
	* Eat a vegetable, it won't kill you.
	* Get up and hit the gym in the morning.
	* Leave the hotel at least once every day.
	* Bring extra charging cords and a powerstrip. Write your name on the supplies you lend out. (@ecarewgrovum)
	* Eat meals with parties of 4. It's much easier to get a table in a busy town. For big groups, just get drinks.
	* Split into smaller groups, if you take a large party to dinner, and you have to wait forever. (@chrislkeller)

	#!/bin/sh
	curl http://cat.www.$1.com.meowbify.com/ > index.html
	s3cmd put index.html s3://$1.cat/

	#!/usr/bin/env ruby
	#
	# This hook can be found in https://github.com/juliamae/githooks

	commit_msg_file = ARGV[0]
	branch_name = `git branch \| grep ''`.gsub(/[ \|\n]/,"")

	if message = open(commit_msg_file).read
	open(commit_msg_file, 'w') do \|f\|
	f.puts message.gsub("Merge", ":part_alternation_mark:erge")