Skip to content

Instantly share code, notes, and snippets.

View johnynek's full-sized avatar

P. Oscar Boykin johnynek

View GitHub Profile
@johnynek
johnynek / TreeList.scala
Created December 14, 2018 19:45
Implementation of "Purely Functional Random Access Lists" by Chris Okasaki in scala. This gives O(1) cons and uncons, and 2 log_2 N lookup.
package org.bykn.list
import cats.Applicative
import cats.implicits._
/**
* Implementation of "Purely Functional Random Access Lists" by Chris Okasaki.
* This gives O(1) cons and uncons, and 2 log_2 N lookup.
*/
@johnynek
johnynek / defer_stack_overflow
Created October 30, 2018 07:50
Hit a stack overflow in Eval.defer in cats 1.4.0
[info] at cats.Eval$.loop$1(Eval.scala:367)
[info] at cats.Eval$.cats$Eval$$evaluate(Eval.scala:372)
[info] at cats.Eval$Defer.value(Eval.scala:258)
[info] at cats.Eval$.loop$1(Eval.scala:351)
[info] at cats.Eval$.cats$Eval$$evaluate(Eval.scala:372)
[info] at cats.Eval$Defer.value(Eval.scala:258)
[info] at cats.Eval$.loop$1(Eval.scala:351)
[info] at cats.Eval$.cats$Eval$$evaluate(Eval.scala:372)
[info] at cats.Eval$Defer.value(Eval.scala:258)
[info] at cats.Eval$.loop$1(Eval.scala:351)
--- /Users/oscar/oss/rules_scala/scala/private/rule_impls.bzl (original)
+++ /Users/oscar/oss/rules_scala/scala/private/rule_impls.bzl (reformatted)
@@ -99,9 +99,7 @@
outs = [ctx.outputs.jar, ctx.outputs.statsfile]
- inputs = ctx.files.resources + [
- ctx.outputs.manifest, zipper_arg_path
- ]
+ inputs = ctx.files.resources + [ctx.outputs.manifest, zipper_arg_path]
@johnynek
johnynek / immutable vs mutable parser.txt
Last active August 6, 2018 03:26
comparing an immutable based parser combinator to a mutable based on that threads a mutable state through during a parse. Probably many optimizations to make on both, which are both very simple.
immutable: timeParsing
mutable: timeParsing2
fastparse: timeParsing3
[info] Benchmark (depth) (length) Mode Cnt Score Error Units
[info] JsonBenchmark.timeParsing 0 5 thrpt 4 1647.064 ± 144.773 ops/s
[info] JsonBenchmark.timeParsing 0 25 thrpt 4 1662.836 ± 23.331 ops/s
[info] JsonBenchmark.timeParsing 0 125 thrpt 4 1675.115 ± 45.713 ops/s
[info] JsonBenchmark.timeParsing 1 5 thrpt 4 273.196 ± 12.285 ops/s
[info] JsonBenchmark.timeParsing 1 25 thrpt 4 37.293 ± 3.452 ops/s
package gradfn
import shapeless.{HNil, :: => HCons, HList, Nat}
import shapeless.ops.nat.ToInt
import Nat.{_0, _1, _2}
import Vect.VectOps
trait NatTrans[F[_], G[_]] {
def apply[A](fn: F[A]): G[A]
// something like this, I didn't actually compile it.
def tailRecM[A, B](a: A)(fn: A => Resource[F, Either[A, B]]): Resource[F, B] = {
def step(adis: (A, F[Unit])): F[Either[(A, F[Unit]), (B, F[Unit])]] = {
val (a, dispose) = dis
val next: F[(Either[A, B], F[Unit])] = fn(a).allocate
// todo: this might not be stack safe, we might
// need a tailRecM type thing on bracket.
def compDisp(d: F[Unit]): F[Unit] =
F.bracket(d)(F.pure)(_ => dispose)
next.map {
@johnynek
johnynek / scala path dependent serializers.scala
Last active July 17, 2018 21:16
Scala's path dependent types can be used to prove that a serialized value can be deserialized without having to resort to Try/Either/Option. This puts the serialized value into the type, so we can be sure we won't fail. This is very useful for distributed compute settings such as scalding or spark.
import scala.util.Try
object PathSerializer {
trait SerDe[A] {
// By using a path dependent type, we can be sure can deserialize without wrapping in Try
type Serialized
def ser(a: A): Serialized
def deser(s: Serialized): A
// If we convert to a generic type, in this case String, we forget if we can really deserialize
package test;
class A {
}
@johnynek
johnynek / dep_clean.py
Created January 24, 2018 17:17
Tool to remove unused deps by brute force on bazel. Stolen shamelessly from Andrew Buss
#!/usr/bin/env python
import sys
# on osx you want brew install gnu-sed to get gsed
from sh import cp, rm, gsed, bazel, ErrorReturnCode
test_fn = lambda: bazel('build', sys.argv[1])
filename = sys.argv[2]
print "Checking that tests pass before starting"
print test_fn()

Simple Bazel Rule for Graphviz

Use rules_nixpkgs to get graphviz:

Make sure you have nix-build installed from nix then add the following to WORKSPACE:

http_archive(
    name = "io_tweag_rules_nixpkgs",
    strip_prefix = "rules_nixpkgs-53700e429928530f1566cfff3ec00283a123f17f",
 urls = ["https://github.com/tweag/rules_nixpkgs/archive/53700e429928530f1566cfff3ec00283a123f17f.tar.gz"],