Skip to content

Instantly share code, notes, and snippets.

View dhalperi's full-sized avatar

Dan Halperin dhalperi

View GitHub Profile
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
.tab:before {
content: "\00bb\00a0\00a0\00a0";
}
.formfeed:before {
content: "\240c";
}
.no-newline:before {
content: "\00a0\2205\21b5";
This file has been truncated, but you can view the full file.
eam.sdk.io.TextSource$TextBasedReader readNextRecord
INFO: DONE RNR
May 05, 2017 7:43:30 PM org.apache.beam.sdk.io.TextSource$TextBasedReader readNextRecord
INFO: RNR
May 05, 2017 7:43:30 PM org.apache.beam.sdk.io.TextSource$TextBasedReader readNextRecord
INFO: DONE RNR
May 05, 2017 7:43:30 PM org.apache.beam.sdk.io.TextSource$TextBasedReader readNextRecord
INFO: RNR
May 05, 2017 7:43:30 PM org.apache.beam.sdk.io.TextSource$TextBasedReader readNextRecord
INFO: DONE RNR
@dhalperi
dhalperi / DateShardedBigQueryTables.java
Created March 30, 2017 15:48
BigQuery example: load an external archive into date-partitioned tables
/*
* Copyright (C) 2016 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
#!/bin/bash
# This is a utility script to merge pull requests into Apache Beam
# repositories. It supports both repositories, source and website.
#
# Usage:
# merge_pr.sh [repository] [branch] pull_request_number
#
# Where:
# repository: { beam | beam-site }
WITH since_incubation AS (
SELECT * FROM `githubarchive.day.201*`
WHERE _TABLE_SUFFIX > '60131' -- after 2016-01-31 since Beam started incubating on 2016-02-01
),
pull_requests AS (
SELECT
-- map a podling to its final name
replace(e.repo.name, "incubator-", "") AS repo,
COUNT(*) AS pr_count
FROM since_incubation AS e
Running io.FirebaseValueTest
[main] INFO io.BaseFirebaseSourceTest - [{"_id":"55e61a3012dc397360c371f2","index":0,"guid":"012118a4-5f74-460d-83f9-acfa91fb1bc3","isActive":false,"balance":"$2,685.88","picture":"http://placehold.it/32x32","age":28,"eyeColor":"blue","name":{"first":"Carrillo","last":"Cantrell"},"company":"ENVIRE","email":"[email protected]","phone":"+1 (923) 418-3207","address":"153 Nassau Street, Choctaw, Tennessee, 3970","about":"Ullamco labore eiusmod fugiat non culpa reprehenderit qui incididunt deserunt nisi minim sit voluptate excepteur. Esse sunt sunt cillum voluptate ea. Incididunt non anim aliquip qui Lorem aute veniam ad cupidatat.","registered":"Tuesday, March 17, 2015 10:20 AM","latitude":"-36.557494","longitude":"164.76669","tags":["mollit","occaecat","et","non","est","deserunt","esse","custom_tag"],"range":[0,1,2,3,4,5,6,7,8,9],"friends":[{"id":0,"name":"Virginia Watkins"},{"id":1,"name":"Duffy Hancock"},{"id":2,"name":"Kelsey Hill"}],"greeting":"Hello, Carrillo! You have
@dhalperi
dhalperi / pelican syntax highlighting example.
Created September 4, 2014 18:06
pelican syntax highlighting example. (python-markdown issue)
Title: Python test
Slug: python-test
Date: 2014-01-20
Tags: python
Python test:
:::python
print("foo")
MyriaStore(public:demo:Cosmo50AllSnapshots)[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaUnionAll[MyriaDupElim[MyriaApply(timestep=1,iOrder=$0,mass=$1,grp=$2,type=$3)[MyriaScan(astro:cosmo50:snapshot512Hash)]], MyriaDupElim[MyriaApply(timestep=2,iOrder=$0,mass=$1,grp=$2,type=$3)[MyriaScan(astro:cosmo50:snapshot504Hash)]]], MyriaDupElim[MyriaApply(timestep=3,iOrder=$0,mass=$1,grp=$2,type=$3)[MyriaScan(astro:cosmo50:snapshot480Hash)]]], MyriaDupElim[MyriaApply(timestep=4,iOrder=$0,mass=$1,grp=$2,type=$3)[MyriaScan(astro:cosmo50:snapshot466Hash)]]], MyriaDupElim[MyriaApply(timestep=5,iOrder=$0,mass=$1,grp=$2,type=$3)[MyriaScan(astro:cosmo50:snapshot456Hash)]]], MyriaDupElim[MyriaApply(timestep=6,iOrder=$0,ma
@dhalperi
dhalperi / typescript
Created May 12, 2014 18:43
script of login/logout to remote linux server
Script started on Mon May 12 11:42:10 2014
]7;file://dhm.dyn.cs.washington.edu/Users/dhalperi%
dhalperi@dhm dhalperi % sssh vea ga
Last login: Mon May 12 11:42:03 2014 from dhm.dyn.cs.washington.edu
Welcome to Ubuntu 13.04 (GNU/Linux 3.8.0-34-generic x86_64)