Jorge Sanz jsanz

gif-from-tweet

There are so many great GIFs out there and I want to have copies of them. Twitter makes that harder than it should be by converting them to MP4 and not providing access to the source material. To make it easier, I made a bash pipeline that takes a tweet URL and a filename, extracts the MP4 from that tweet and uses ffmpeg to convert back to GIF.

Dependencies

ffmpeg
- macOS: brew install ffmpeg
- Ubuntu/Debian: apt install ffmpeg

What follows is a technical test for this job offer at CARTO: https://boards.greenhouse.io/cartodb/jobs/705852#.WSvORxOGPUI

Build the following and make it run as fast as you possibly can using Python 3 (vanilla). The faster it runs, the more you will impress us!

Your code should:

Download this ~2GB file: https://s3.amazonaws.com/carto-1000x/data/yellow_tripdata_2016-01.csv
Count the lines in the file
Calculate the average value of the tip_amount field.

TechChange: Technology for Data Visualization

Basics

Use the CARTO Builder for easy visualization of data
- Put your data on the map
- Build dashboards to interact with data
- Dynamically updating maps (sync'd from realtime data)
We have free accounts: https://carto.com/signup

Explore Time and Space with SQL and PostGIS

Get this doc here: http://bit.ly/sql-postgis-time-space

Get a free CARTO account

We're going to use CARTO's builder. Signup for free accounts here: https://carto.com/signup

We're going to explore the space (through PostGIS) and time (native PostgreSQL) in today's workshops using two datasets that have lat/long and timestamps.

	#!/bin/bash

	BLIGHT_PATH=$(sudo find /sys -type d -name intel_backlight)

	MAX_BRIGHTNESS=$(cat $BLIGHT_PATH/max_brightness)

	while [ 1 ]; do
	echo "$MAX_BRIGHTNESS*$(fswebcam --png -1 - \| convert - -colorspace gray -format "%[fx:mean]" info:)" \| \
	bc -l \| cut -d '.' -f 1 \| \
	sudo tee $BLIGHT_PATH/brightness;

	<html>
	<body>
	<div id="react-root"></div>

	<script src="https://cdnjs.cloudflare.com/ajax/libs/react/16.0.0/umd/react.production.min.js"></script>
	<script src="https://cdnjs.cloudflare.com/ajax/libs/react-dom/16.0.0/umd/react-dom.production.min.js"></script>
	<script src="https://cdnjs.cloudflare.com/ajax/libs/babel-standalone/6.26.0/babel.min.js"></script>

	<script id="react-app" type="text/template">
	const App = ({name}) => {

	import pandas as pd
	from geopandas import GeoDataFrame
	from shapely.geometry import Point
	import fiona

	df = pd.read_csv('data.csv')

	geometry = [Point(xy) for xy in zip(df.x, df.y)]
	crs = {'init': 'epsg:2263'} #http://www.spatialreference.org/ref/epsg/2263/
	geo_df = GeoDataFrame(df, crs=crs, geometry=geometry)

	'use strict';

	var DEFAULTS = {
	endpoint: 'https://rochoa.carto.com',
	sql: 'select * from airbnb_madrid_oct_2015_listings',
	cartocss: `#layer['mapnik::geometry_type'=1] {
	marker-width: 7;
	marker-fill: #EE4D5A;
	marker-fill-opacity: 0.9;
	marker-line-color: #FFFFFF;

	import urllib,sys,csv;
	print reduce(lambda p, new: (p[0] + (new - p[0])/p[1], p[1] + 1), map(float, (x['tip_amount'] for x in csv.DictReader(urllib.urlopen(sys.argv[1])))), (0, 1))