Skip to content

Instantly share code, notes, and snippets.

View erikerlandson's full-sized avatar

Erik Erlandson erikerlandson

View GitHub Profile
@erikerlandson
erikerlandson / commonBuilders.scala
Last active February 14, 2017 21:52
Case classes inheriting common builder methods from base trait
object demo {
trait TLike[T <: TLike[T]] { self: T =>
def x: Int
def y: Int
protected def construct(x: Int, y: Int): T
private def copyTrait(x: Int = self.x, y: Int = self.y): T = construct(x, y)
def xSet(x: Int): T = copyTrait(x = x)
def ySet(y: Int): T = copyTrait(y = y)
}
@erikerlandson
erikerlandson / DelayedMap.scala
Created November 24, 2016 22:36
test harness for simulating long-running Spark jobs
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
package cats
package data
import scala.annotation.tailrec
sealed abstract class Continuation[O, +I] {
def map[I2](fn: I => I2): Continuation[O, I2] =
Continuation.Mapped(this, fn)
def flatMap[I2](fn: I => Continuation[O, I2]): Continuation[O, I2] =
@erikerlandson
erikerlandson / annotate.md
Last active March 6, 2017 21:29
break and continue as monads for Scala for-comprehensions

The concept in this gist works properly, but evaluates eagerly. Also, its type signature is a bit more complicated than necessary, with the self-typing signature trying to preserve the outer sequence type.

I recommend the breakable package, which supports lazy evaluation, along with some other useful features:

https://github.com/erikerlandson/breakable

// 'small' was a subsample of meta-full parquet
scala> small.orderBy(small("@timestamp")).map(utils.toJSON).saveAsTextFile("/home/eje/logdemo/logs_ordered.json")
@erikerlandson
erikerlandson / distplot_bug.html
Last active May 23, 2016 19:58
distplot repro
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html>
<head><meta charset="utf-8" />
<title>distplot_bug</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>
<style type="text/css">
/*!
@erikerlandson
erikerlandson / temp
Last active May 23, 2016 16:31
jupyter notebook repro for plotly distplot bug
xxx
@erikerlandson
erikerlandson / distplot_bug.ipynb
Last active May 23, 2016 16:29
Jupyter notebook reproducing plotly distplot bug
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@erikerlandson
erikerlandson / distplot_bug.ipynb
Last active May 23, 2016 16:27
Jupyter notebook showing bug in plotly distplot
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@erikerlandson
erikerlandson / cluster_1
Last active May 6, 2016 22:34
Record the results of Random Forest clustering on machine package installations
0.0 gss-diag01
0.0 gss-servicejava01
0.0 gss-servicejava02
0.0 gss-servicejava03
0.0 gss-servicejava04
0.0 gss-webjava01
0.0 gss-webjava02
0.0 gss-webjava03
0.0 gss-webjava04