This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Very naive approach to simulating a distributed cache using consistent hashing. Based on the following: | |
* http://thor.cs.ucsb.edu/~ravenben/papers/coreos/kll%2B97.pdf | |
* | |
* Initially a translation of: | |
* http://weblogs.java.net/blog/tomwhite/archive/2007/11/consistent_hash.html | |
* | |
* And I also looked at this gist a bit from http://github.com/opyate towards the end as well: | |
* https://gist.github.com/1927001 | |
* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"/hello/:name" get { | |
request => Future { | |
/* | |
* we're in The Future here so you can do | |
* heavier stuff without blocking up other | |
* things like the Netty handler. | |
*/ | |
val name = request.path(":name") | |
Response.ok(s"Hello, ${name}") | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package definterp | |
import org.scalatest._ | |
class AdditionProgram extends FlatSpec with Matchers { | |
val addition = | |
Lambda("x", | |
Lambda("y", LetRec("add-until", | |
Lambda("xx", |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/** | |
* Started to rough this naive S3-native filesystem RDD out because I need to use IAM | |
* profiles for S3 access and also https://issues.apache.org/jira/browse/HADOOP-3733. | |
* | |
* Use at your own risk, bear in mind this is maybe 30 - 45min of work and testing and | |
* expect it to behave as such. | |
* | |
* Feedback/criticism/discussion welcome via Github/Twitter | |
* | |
* In addition to Spark 1.0.x, this depends on Amazon's S3 SDK, dependency is as follows: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val databaseUrl = "postgresql://some-hostname:5432/db-name" | |
Class.forName("my.sql.database.driver.classname") | |
class BasicJdbcActor(connFac: () => Connection) extends Actor { | |
lazy val conn = connFac() | |
override def preRestart(why: Throwable, msg: Option[Any]): Unit = | |
try { conn.close() } | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
my-dispatcher { | |
type = Dispatcher | |
executor = "fork-join-executor" | |
fork-join-executor { | |
parallelism-min = 2 | |
//2 threads per core | |
parallelism-factor = 2.0 | |
// The max that the dispatcher will create: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// very naive, be more specific based on your problem: | |
val restartStrategy = OneForOneStrategy( | |
maxNrOfRetries = 10, | |
withinTimeRange = 1 minute) { | |
case _ => Restart | |
} | |
def newPool(sys: ActorSystem): ActorRef = { | |
val props = Props(new BasicJdbcActor(connFac)) | |
val pool = RoundRobinPool(4, supervisorStrategy = restartStrategy) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val props = Props(new BasicJdbcActor(connFac)) | |
.withDispatcher("my-dispatcher") | |
val pool = RoundRobinPool(4, supervisorStrategy = restartStrategy) | |
sys.actorOf(pool.props(props)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
case class Person(name: String, email: String) | |
case class PersonById(id: Int) | |
class PersonDao(cf: () => Connection) extends Actor { | |
lazy val conn = cf() | |
override def preRestart(why: Throwable, msg: Option[Any]): Unit = | |
try { conn.close() } | |
def receive = { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
trait PersonClient { | |
// supply a router with a pool of PersonDao: | |
val personPool: ActorRef | |
// how long should we wait for a response from PersonDao: | |
val timeoutInMillis: Long | |
implicit val timeout = Timeout(timeoutInMillis millis) | |
def addPerson(p: Person): Future[Int] = |
OlderNewer