Skip to content

Instantly share code, notes, and snippets.

View codejitsu's full-sized avatar

Alex codejitsu

View GitHub Profile
@runarorama
runarorama / gist:33986541f0f1ddf4a3c7
Created May 7, 2015 14:06
Higher-kinded types encoded as path-dependent types
trait λ {
type α
}
trait Functor extends λ {
type α <: λ
def map[A,B](x: α { type α = A })(f: A => B): α { type α = B }
}
@rbranson
rbranson / gist:038afa9ad7af3693efd0
Last active September 29, 2016 17:44
Disaggregated Proxy & Storage Nodes

The point of this is to use cheap machines with small/slow storage to coordinate client requests while dedicating the machines with the big and fast storage to doing what they do best. I found that request coordination was contributing to about half the CPU usage on our Cassandra nodes, on average. Solid state storage is quite expensive, nearly doubling the cost of typical hardware. It also means that if people have control over hardware placement within the network, they can place proxy nodes closer to the client without impacting their storage footprint or fault tolerance characteristics.

This is accomplished in Cassandra by passing the -Dcassandra.join_ring=false option when the process is started. These nodes will connect to the seeds, cache the gossip data, load the schema, and begin listening for client requests. Messages like "/x.x.x.x is now UP!" will appear on the other nodes.

There are also some more practical benefits to this. Handling client requests caused us to push the NewSize of the heap up

Actor Queue Notes

The first thing to understand is that the head field inside of scalaz.concurrent.Actor is not the "head" of the message queue in any traditional sense of the word. A better description would be "last". The there are no pointers to the head of the queue, which one of the very clever things about this implementation.

Empty Actor

Consider the case where the actor has no outstanding messages. This new message will go into the following code:

 def !(a: A): Unit = {
@squito
squito / AccumulatorListener.scala
Last active March 15, 2019 06:34
Accumulator Examples
import scala.collection.mutable.Map
import org.apache.spark.{Accumulator, AccumulatorParam, SparkContext}
import org.apache.spark.scheduler.{SparkListenerStageCompleted, SparkListener}
import org.apache.spark.SparkContext._
/**
* just print out the values for all accumulators from the stage.
* you will only get updates from *named* accumulators, though
// Define the following traits and companion object
// It's in Rapture Core (https://github.com/propensive/rapture-core) if you don't want to
trait LowPriorityDefaultsTo { implicit def fallback[T, S]: DefaultsTo[T, S] = null }
object DefaultsTo extends LowPriorityDefaultsTo { implicit def defaultDefaultsTo[T]: DefaultsTo[T, T] = null }
trait DefaultsTo[T, S]
// Then, assuming we want to specify a default for a type class like `Namer`,
case class Namer[T](name: String)
@killercup
killercup / Readme.md
Last active August 26, 2023 23:14
Convert The Rust Programming Language to Epub

webapp/OAuth2/REST API stack security checklist

OAuth2 (implicit flow)

  • response_type is a constant set for particular app on server (1)
  • redirect_uri, too (1)
  • "state" parameter is echoed by server and checked by client (1)
  • hash of "state" parameters is stored in LocalStorage and destroyed on receiving redirect (1)
  • "client_id" parameter is checked (after obtaining token, make a request to server to get "token_id" for this token and compare to existing one) (2, 3)
  • IP is saved for particular token on grant and checked on access
@jastice
jastice / lebkuchen.md
Last active August 29, 2015 14:10
Justin's Lebkuchen

Justin's Lebkuchen

A variation of Elisenlebkuchen, which contain no flour. Makes ca 45-60 Lebkuchen, 7cm.

Vegan, gluten-free (with the right kind of oblaten), certainly not low-fat or low-sugar.

Primary ingredients

  • 900g nuts or similar. Preferred mixture:
@rklaehn
rklaehn / Client example
Last active June 8, 2020 12:38
akka http file server
package akkahttptest
import akka.http.Http
import akka.stream.ActorFlowMaterializer
import akka.actor.ActorSystem
import akka.stream.scaladsl.{Sink, Source}
import akka.http.model._
object TestClient extends App {