Skip to content

Instantly share code, notes, and snippets.

View calvinlfer's full-sized avatar

Cal calvinlfer

View GitHub Profile
@calvinlfer
calvinlfer / server.scala
Created November 16, 2017 22:44 — forked from huntc/server.scala
A complete server using Akka streams that reads some source, batches its data and then publishes. If the data cannot be published then it backs off with a best-effort of sending that data again.
val (recycleQueue, recycleSource) =
Source
.queue[SoilStateReading](100, OverflowStrategy.dropTail)
.prefixAndTail(0)
.map(_._2)
.toMat(Sink.head)(Keep.both)
.run()
StreamConverters.fromInputStream(() => this.getClass.getClassLoader.getResourceAsStream("sensors.log"))
.via(SoilStateReading.csvParser)
.merge(Source.fromFutureSource(recycleSource))

Quick Tips for Fast Code on the JVM

I was talking to a coworker recently about general techniques that almost always form the core of any effort to write very fast, down-to-the-metal hot path code on the JVM, and they pointed out that there really isn't a particularly good place to go for this information. It occurred to me that, really, I had more or less picked up all of it by word of mouth and experience, and there just aren't any good reference sources on the topic. So… here's my word of mouth.

This is by no means a comprehensive gist. It's also important to understand that the techniques that I outline in here are not 100% absolute either. Performance on the JVM is an incredibly complicated subject, and while there are rules that almost always hold true, the "almost" remains very salient. Also, for many or even most applications, there will be other techniques that I'm not mentioning which will have a greater impact. JMH, Java Flight Recorder, and a good profiler are your very best friend! Mea

@calvinlfer
calvinlfer / example.scala
Created January 15, 2018 20:24
Phantom types example: Detect passwords that are plain vs. hashed
import com.github.t3hnar.bcrypt._
// Phantom type used in detecting questions and credentials using plain or hashed passwords
sealed trait PasswordStatus
trait Plain extends PasswordStatus
trait Hashed extends PasswordStatus
case class SecretQuestion[Status <: PasswordStatus](memberId: UUID, questionId: String, answer: String)
object SecretQuestion {
@calvinlfer
calvinlfer / distribute.scala
Created January 16, 2018 04:29
A functional implementation of distributing Auto-Scaling-Groups to different AZs in the us-east-1 region
import scala.annotation.tailrec
sealed trait UsEast1AvailabilityZone
case object UsEast1a extends UsEast1AvailabilityZone
case object UsEast1b extends UsEast1AvailabilityZone
case object UsEast1c extends UsEast1AvailabilityZone
type Count = Int
def distribute(nrOfAsgs: Int, regions: List[UsEast1AvailabilityZone]): Map[UsEast1AvailabilityZone, Count] = {
@tailrec
@calvinlfer
calvinlfer / build.sbt
Created January 23, 2018 04:22
Marshalling and unmarshalling case classes to and from JSON with Akka HTTP Circe
scalaVersion := "2.12.4"
libraryDependencies ++= {
val akkaHttpV = "10.0.11"
val circeV = "0.9.1"
Seq(
"com.typesafe.akka" %% "akka-http" % akkaHttpV,
"io.circe" %% "circe-core" % circeV,
"io.circe" %% "circe-generic" % circeV,
"io.circe" %% "circe-parser" % circeV,
@calvinlfer
calvinlfer / example.scala
Created February 8, 2018 23:58
Racing futures (bounding the wait time of a Future using a timeout) with Akka
import akka.actor.ActorSystem
import akka.pattern.after
import scala.concurrent.{ExecutionContext, Future}
import scala.concurrent.duration._
val system: ActorSystem = ActorSystem()
implicit val ec: ExecutionContext = system.dispatcher
val fallback: Future[Option[Int]] = after(2.seconds, system.scheduler)(Future.successful(None))
@calvinlfer
calvinlfer / script.sh
Last active March 2, 2022 06:52
Pushing Jenkins ThinBackup backups to S3
#!/bin/bash -l
# find latest backup
thinbackup_path="$JENKINS_HOME/thinBackup"
latest_backup_folder=$(ls -t $thinbackup_path | head -n1)
full_path="$thinbackup_path/$latest_backup_folder"
echo "Latest backup is located at ${full_path}"
# pack it up
backup_name="$latest_backup_folder.tar.gz"
@calvinlfer
calvinlfer / AmmoniteFun.scala
Created July 28, 2018 04:52
Copying files in parallel on S3 with Ammonite
import java.time.{ZoneId, ZonedDateTime}
import java.time.format.DateTimeFormatter
import ammonite.ops._
import scala.concurrent._
import duration._
object AmmoniteFun extends App {
implicit val path: Path = pwd
@calvinlfer
calvinlfer / FPtoTheMax.scala
Last active August 9, 2018 19:15
Following along with FP to the max (https://www.youtube.com/watch?v=sxudIMiOo68)
import scala.io.StdIn
import scala.language.higherKinds
import scala.util.Try
object Practice extends App {
trait Program[F[_]] {
def finish[A](a: => A): F[A]
def chain[A, B](fa: F[A], afb: A => F[B]): F[B]
@calvinlfer
calvinlfer / fs2-take.scala
Last active August 13, 2018 03:08
A simple implementation of take for FS2 streams to familiarize myself with the Pull API
import fs2._
def myTake[F[_], A](nrOfElements: Int): Pipe[F, A, A] = inputStream => {
def go(s: Stream[F, A], remaining: Int): Pull[F, A, Unit] =
if (remaining == 0) Pull.done
else s.pull.uncons1.flatMap {
case Some((element: A, remStream: Stream[F, A])) => Pull.output1(element) >> go(remStream, remaining - 1)
case None => Pull.done
}