Side note: all available resource metrics are documented here:
These are used for isolating files on disk from both the host system as well as other running tasks.
Alexander Gallego emaxerrno
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package com.yieldmo.rtb.util | |
| class LazyMap[A,B](delegate: Map[A,() => B]) extends scala.collection.immutable.Map[A,B]{ | |
| def get(key: A): Option[B] = delegate.get(key).map{ _() } | |
| def iterator: Iterator[(A,B)] = delegate.iterator.map{ pair => (pair._1, pair._2()) } | |
| def +[B1 >:B](kv: (A, B1)): Map[A, B1] = new LazyMap[A,B1](delegate + (kv._1 -> (() => kv._2))) | |
| def -(key: A): Map[A, B] = new LazyMap[A,B](delegate - key) | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Producer | |
| Setup | |
| bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test-rep-one --partitions 6 --replication-factor 1 | |
| bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test --partitions 6 --replication-factor 3 | |
| Single thread, no replication | |
| bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance test7 50000000 100 -1 acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 buffer.memory=67108864 batch.size=8196 |
johnynek
/ AliceInAggregatorLand.scala
Last active
January 24, 2024 19:38
A REPL Example of using Aggregators in scala
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * To get started: | |
| * git clone https://github.com/twitter/algebird | |
| * cd algebird | |
| * ./sbt algebird-core/console | |
| */ | |
| /** | |
| * Let's get some data. Here is Alice in Wonderland, line by line | |
| */ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ... | |
| NiftySecurityHandlers niftySecurityHandlers = new NiftySecurityHandlers() { | |
| @Override | |
| public ChannelHandler getAuthenticationHandler() { | |
| return noOpHandler; | |
| } | |
| @Override | |
| public ChannelHandler getEncryptionHandler() { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Instrument binaries, pgo data to /data/pgo, serial make is important to not confuse the pgo generator | |
| env CXXFLAGS='-march=native -fprofile-dir=/data/pgo -fprofile-generate=/data/pgo' cmake .. -DCMAKE_BUILD_TYPE=Release | |
| make -j 1 | |
| # Run instrumented program, generate and write pgo data | |
| ./runIt | |
| # Use profile data and feed into gcc, correct for threading counter noise, serial make is important to not confuse the pgo generator | |
| env CXXFLAGS='-march=native -fprofile-dir=/data/pgo -fprofile-use=/data/pgo -fprofile-correction' cmake .. -DCMAKE_BUILD_TYPE=Release | |
| make -j 1 |
chetan
/ mesos_isolators.md
Last active
January 18, 2019 21:55
Description of available Apache Mesos isolators
OlderNewer