Skip to content

Instantly share code, notes, and snippets.

@zedar
zedar / ApacheHadoopSpark_LZO.adoc
Last active April 24, 2024 14:33
Add LZO compression codecs to the Apache Hadoop and Spark

Add LZO compresssion codecs to the Apache Hadoop and Spark

LZO is a splittable compression format for files stored in Hadoop’s HDFS. It has valuable combination of speed and compression size. Thanks to hadoop-lzo the .lzo files could be splittable too.

  • Install lzo and lzop codes [OSX].

$ brew install lzo lzop
  • Find where the headers and libraries are installed

Explaining Miles's Magic

Miles Sabin recently opened a pull request fixing the infamous SI-2712. First off, this is remarkable and, if merged, will make everyone's life enormously easier. This is a bug that a lot of people hit often without even realizing it, and they just assume that either they did something wrong or the compiler is broken in some weird way. It is especially common for users of scalaz or cats.

But that's not what I wanted to write about. What I want to write about is the exact semantics of Miles's fix, because it does impose some very specific assumptions about the way that type constructors work, and understanding those assumptions is the key to getting the most of it his fix.

For starters, here is the sort of thing that SI-2712 affects:

def foo[F[_], A](fa: F[A]): String = fa.toString
@mfikes
mfikes / fast.md
Last active January 6, 2024 07:19
Fast Clojure REPL

An experimental change for fast Clojure REPL startup:

  1. Download the JAR: clojure-1.8.0-fast.jar
  2. Launch it via java -jar clojure-1.8.0-fast.jar

The code used to create this JAR is on GitHub.

What's it doing?

It is:

@OlegIlyenko
OlegIlyenko / CustomJsonScalarType.scala
Last active March 4, 2022 16:48
An example of custom raw JSON scalar type in sangria. DON'T USE IT! By using it you lose many benefits of GraphQL. This just demonstrates that it is possible. If you tempted to expose it, then definitely think twice before using it.
import sangria.ast
import sangria.execution.Executor
import sangria.marshalling.{InputUnmarshaller, ScalarValueInfo, ArrayMapBuilder, ResultMarshaller}
import sangria.schema._
import sangria.validation.{ValueCoercionViolation, IntCoercionViolation, BigIntCoercionViolation}
import spray.json._
import sangria.macros._
import scala.concurrent.ExecutionContext.Implicits.global
implicit object CustomSprayJsonResultMarshaller extends ResultMarshaller {
@ikhoon
ikhoon / must-watch-talks.md
Last active April 14, 2020 10:46
Must watch talks - reddit /r/scala
@tomfa
tomfa / BeanstalkViaTerraform
Last active March 15, 2022 07:51
Adding Beanstalk roles with Terraform
resource "aws_iam_instance_profile" "beanstalk_service" {
name = "beanstalk-service-user"
roles = ["${aws_iam_role.beanstalk_service.name}"]
}
resource "aws_iam_instance_profile" "beanstalk_ec2" {
name = "beanstalk-ec2-user"
roles = ["${aws_iam_role.beanstalk_ec2.name}"]
}

Welcome to Phalka Posse!

Mission: Create intentional independet positive change

Phalka at Turing:

  • Tuesdays: come eat lunch
  • Fridays: Spikes will be dedicated to project work time or speakers or panels

Phalka outside of Turing

  • As these events come up you'll be notified
@klpx
klpx / splitEither.scala
Last active June 18, 2019 16:25
Akka Streams. Graph for Split Either[L,R] to L and R flows
/**
Copy left Alexander Hasselbach
Usage:
val E = b.add(splitEither[Throwable,Int])
val parsed = b.add(Flow[Either[Throwable,Int]])
val valids = b.add(Flow[Int])
val invalids = b.add(Flow[Throwable])