| Steps to verify our Kafka Mirror App works: | |
| # Start Kafka | |
| docker run -p 2181:2181 -p 9092:9092 \ | |
| --name kafka --rm \ | |
| --env ADVERTISED_HOST=172.17.0.2 \ | |
| --env ADVERTISED_PORT=9092 \ | |
| spotify/kafka | |
| # Start producer |
You don't need to spell out the whole case class when you're doing transformations.
If your data model is changing quite a lot except for a few key parts that your application is interested in - it shouldn't even be aware of the other stuff.
Here we use play-json format composition to read the whole JsObject into one field of the case class - and write it back out.
| def fib(n): | |
| pass | |
| assert fib(0) == 0 | |
| assert fib(1) == 1 | |
| assert fib(2) == 1 | |
| assert fib(3) == 2 | |
| assert fib(4) == 3 | |
| assert fib(5) == 5 | |
| assert fib(6) == 8 |
| Jan 21 13:08:02 [91.179.173.244] ~|TAA|~Shadow says: 'Thanks!' | |
| Jan 21 13:08:29 [141.196.219.246] AngelOfDeath(* says: 'Thanks, man' | |
| Jan 21 13:08:34 [91.179.173.244] ~|TAA|~Shadow says: 'Thanks, man' | |
| Jan 21 13:08:52 [91.179.173.244] ~|TAA|~Shadow says: 'Thanks!' | |
| Jan 21 13:09:12 [91.179.173.244] ~|TAA|~Shadow says: 'Thanks, man' | |
| Jan 21 13:09:21 [141.196.219.246] AngelOfDeath(* says: 'Thanks!' | |
| Jan 21 13:09:34 [91.179.173.244] ~|TAA|~Shadow says: 'Thanks, man' | |
| Jan 21 13:09:39 [91.179.173.244] ~|TAA|~Shadow says: 'Thanks, man' | |
| Jan 21 13:10:00 [91.179.173.244] ~|TAA|~Shadow says: 'Thanks, man' | |
| Jan 21 13:10:10 [91.179.173.244] ~|TAA|~Shadow says: 'Thanks, man' |
| 2015-12-01 Please consider spending 5 minutes every day in our <a href="https://groups.google.com/forum/#!forum/actionfps">new Community Forum</a>. Your participation, activity and feedback is always welcome. | |
| 2016-01-01 | |
| 2016-01-03 http://actionfps.tumblr.com/post/134415507169/a-test | |
| 2016-03-30 Now we list <a href="https://actionfps.com/playerranks/">all players in the player ranks</a> regardless of their rank. | |
| 2016-04-01 Added the <a href="/hof/">Hall of Fame</a>! Should we have a prize for the first Cube Addict?<br/> Tell us <a href="https://gitter.im/ScalaWilliam/ActionFPS">on the ActionFPS Gitter chatroom</a>. | |
| 2016-04-10 "Added a new server <a href=""assaultcube://califa.actionfps.com:1999"">Califa.ActionFPS.Com:1999</a> (Los Angeles, USA)<br/>Added a new server <a | |
| href=""assaultcube://bonza.actionfps.com:1999"">Bonza.ActionFPS.Com:1999</a> (Sydney, Australia)<br/>Added a new server <a | |
| href=""assaultcube://legal.actionfps.com:1999"">Legal.ActionFPS.Com:1999</a> (São Paulo, Brazil)" | |
| 2016-04-22 Added the |
| import java.io.File | |
| import java.util.concurrent.TimeUnit | |
| import okhttp3.{Cache, CacheControl, OkHttpClient, Request} | |
| import org.apache.http.impl.client.cache.{CacheConfig, CachingHttpClientBuilder} | |
| object ExpApp extends App { | |
| def ok(): Unit = { |
| ~ curl https://actionfps.com/all/games.tsv > /dev/null | |
| % Total % Received % Xferd Average Speed Time Time Time Current | |
| Dload Upload Total Spent Left Speed | |
| 100 15.0M 0 15.0M 0 0 856k 0 --:--:-- 0:00:18 --:--:-- 1726k | |
| ~ curl https://actionfps.com/all/games.tsv > /dev/null | |
| % Total % Received % Xferd Average Speed Time Time Time Current | |
| Dload Upload Total Spent Left Speed | |
| 100 15.0M 0 15.0M 0 0 1425k 0 --:--:-- 0:00:10 --:--:-- 2501k |
Demonstration of a file based concurrent message queue with the JVM (using Nashorn in this case).
We use simple file locking to write to the file. I managed to get 25k messages/s in doing this on Mac OS X which is plenty fast for most projects.
Notice how read.js is calling into a process. This means we can call into an external SSH server very easily to consume events.
They can be consumed by ANY language or client we want. Bash, Python, awk, Scala, Java, Perl, JavaScript, ...
write-many.js demonstrates how to continuously append. This is not batched so likely to be much slower.
Run two Scala consoles with this, one of them should block.
Then do lock.release() and the other will unlock.
I tested this on Mac OS X 10.11, CentOS 7, Windows 10.
This can be used to implement a multiple-writers & multiple tailers mechanism ie for a very simple shared message queue.
| #!/usr/bin/perl | |
| while ($line = <>) { | |
| if ( $line =~ /Status at (\d\d)-(\d\d)-(\d\d\d\d) (\d\d):(\d\d):(\d\d):/ ) { | |
| $last_date = "$3-$2-$1"; | |
| $last_time = "$4:$5:$6"; | |
| } | |
| if ( $last_date && $line =~ /^[A-Z][a-z][a-z] \d\d (.*)/ ) { | |
| print $last_date ."T". $1."\n"; | |
| } elsif ( $last_date && $last_date =~ /^(2016|2017-01)/ ) { | |
| print $last_date . "T" . $last_time . " ". $line; |