Felipe Forbeck fforbeck

Getting Started in Scala

This is my attempt to give Scala newcomers a quick-and-easy rundown to the prerequisite steps they need to a) try Scala, and b) get a standard project up and running on their machine. I'm not going to talk about the language at all; there are plenty of better resources a google search away. This is just focused on the prerequisite tooling and machine setup. I will not be assuming you have any background in JVM languages. So if you're coming from Python, Ruby, JavaScript, Haskell, or anywhere… I hope to present the information you need without assuming anything.

Disclaimer It has been over a decade since I was new to Scala, and when I was new to Scala, I was coming from a Java and Ruby background. This has probably caused me to unknowingly make some assumptions. Please feel free to call me out in comments/tweets!

One assumption I'm knowingly making is that you're on a Unix-like platform. Sorry, Windows users.

Getting the JVM

AWS Service annotations

service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval (in minutes)
service.beta.kubernetes.io/aws-load-balancer-access-log-enabled (true|false)
service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name
service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix
service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags (comma-separated list of key=value)
service.beta.kubernetes.io/aws-load-balancer-backend-protocol (http|https|ssl|tcp)
service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled (true|false)

Safest way to clean up boot partition - Ubuntu 14.04LTS-x64, Ubuntu 16.04LTS-x64

Reference

Case I: if /boot is not 100% full and apt is working

1. Check the current kernel version

$ uname -r

##Reactive System Design Links

#Articles and Papers

The Reactive Manifesto: http://www.reactivemanifesto.org/
https://en.wikipedia.org/wiki/Reactive_programming
https://en.wikipedia.org/wiki/Functional_reactive_programming
Design Methods for Reactive Systems book slides: http://booksite.elsevier.com/9781558607552/slides/slides.pdf
Programming without a callstack Event Driven Architectures By G Hohpe: http://www.eaipatterns.com/docs/EDA.pdf
On Distributed Memory Systems: http://blog.paralleluniverse.co/2012/07/10/on-distributed-memory/
Disruptor source code, papers and articles: https://lmax-exchange.github.io/disruptor/

My efforts to port http://www.youtube.com/watch?v=f6kdp27TYZs to Clojure.

A boring function (6:22)

func boring(msg string) {
  for i := 0; ; i++ {
    fmt.Println(msg, i)
    time.Sleep(time.Second)
 }

	{:user {:dependencies [[org.clojure/tools.namespace "0.2.3"]
	[spyscope "0.1.3"]
	[criterium "0.4.1"]]
	:injections [(require '(clojure.tools.namespace repl find))
	; try/catch to workaround an issue where `lein repl` outside a project dir
	; will not load reader literal definitions correctly:
	(try (require 'spyscope.core)
	(catch RuntimeException e))]
	:plugins [[lein-pprint "1.1.1"]
	[lein-beanstalk "0.2.6"]

	// installed Clojure packages:
	//
	// * BracketHighlighter
	// * lispindent
	// * SublimeREPL
	// * sublime-paredit

	{
	"word_separators": "/\\()\"',;!@$%^&\|+=[]{}`~?",
	"paredit_enabled": true,

	DECLARE E INT DEFAULT 0;
	DECLARE M TEXT DEFAULT NULL;

	DECLARE CONTINUE HANDLER FOR 1000 SET E='1000', M="hashchk";
	DECLARE CONTINUE HANDLER FOR 1001 SET E='1001', M="isamchk";
	DECLARE CONTINUE HANDLER FOR 1002 SET E='1002', M="NO";
	DECLARE CONTINUE HANDLER FOR 1003 SET E='1003', M="YES";
	DECLARE CONTINUE HANDLER FOR 1004 SET E='1004', M="Can't create file '%s' (errno: %d)";
	DECLARE CONTINUE HANDLER FOR 1005 SET E='1005', M="Can't create table '%s' (errno: %d)";
	DECLARE CONTINUE HANDLER FOR 1006 SET E='1006', M="Can't create database '%s' (errno: %d)";

	Latency Comparison Numbers (~2012)
	----------------------------------
	L1 cache reference 0.5 ns
	Branch mispredict 5 ns
	L2 cache reference 7 ns 14x L1 cache
	Mutex lock/unlock 25 ns
	Main memory reference 100 ns 20x L2 cache, 200x L1 cache
	Compress 1K bytes with Zippy 3,000 ns 3 us
	Send 1K bytes over 1 Gbps network 10,000 ns 10 us
	Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD

	If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example:

	- Use create in the index API (assuming you can).
	- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval).
	- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap.
	- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000).
	- Increase the memory allocated to elasticsearch node. By default its 1g.
	- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine.
	- Increase the number of machines you have so