This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# bash generate random alphanumeric string | |
# | |
# bash generate random 32 character alphanumeric string (upper and lowercase) and | |
NEW_UUID=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1) | |
# bash generate random 32 character alphanumeric string (lowercase only) | |
cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
18/08/28 07:01:13 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |
Loading required package: methods | |
Attaching package: ‘SparkR’ | |
The following objects are masked from ‘package:stats’: | |
cov, filter, lag, na.omit, predict, sd, var, window | |
The following objects are masked from ‘package:base’: | |
as.data.frame, colnames, colnames<-, drop, endsWith, intersect, | |
rank, rbind, sample, startsWith, subset, summary, transform, union | |
Warning message: | |
package ‘SparkR’ was built under R version 3.4.4 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
15/05/25 11:52:22 INFO org.apache.spark.SecurityManager: Changing view acls to: prashant | |
15/05/25 11:52:22 INFO org.apache.spark.SecurityManager: Changing modify acls to: prashant | |
15/05/25 11:52:22 INFO org.apache.spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(prashant); users with modify permissions: Set(prashant) | |
15/05/25 11:52:22 ERROR org.apache.spark.executor.Executor: Exception in task 3.0 in stage 6.0 (TID 27) | |
java.lang.ExceptionInInitializerError | |
at $line31.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<console>:20) | |
at $line31.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<console>:25) | |
at $line31.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(<console>:27) | |
at $line31.$read$$iw$$iw$$iw$$iw$$iw.<init>(<console>:29) | |
at $line31.$read$$iw$$iw$$iw$$iw.<init>(<console>:31) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
prashant@sc:~/work/spark-examples$ mvn dependency:tree | |
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 | |
[INFO] Scanning for projects... | |
[INFO] | |
[INFO] ------------------------------------------------------------------------ | |
[INFO] Building Spark Project Examples 0.1.0-SNAPSHOT | |
[INFO] ------------------------------------------------------------------------ | |
[INFO] | |
[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ spark-examples_2.10 --- | |
[INFO] org.apache.spark:spark-examples_2.10:jar:0.1.0-SNAPSHOT |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
scala> import scala.reflect._ | |
import scala.reflect._ | |
scala> def f[K: ClassTag, V: ClassTag, T<:Product2[K, V]](names: T, values: T) = { | |
val x = values; | |
println(classTag[K]); | |
names.productIterator.toList.zip(x.productIterator.map(_.toString).toSeq).map(kv => kv._1+"="+kv._2) | |
} | |
| | | | f: [K, V, T <: Product2[K,V]](names: T, values: T)(implicit evidence$1: scala.reflect.ClassTag[K], implicit evidence$2: scala.reflect.ClassTag[V])List[String] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import akka.actor.{Props, ActorRef, Actor, ActorSystem} | |
import com.typesafe.config.ConfigFactory | |
import scala.util.Random | |
import scala.collection.mutable.LinkedList | |
import scala.reflect.ClassTag | |
import scala.collection.mutable | |
import akka.remote.RemoteActorRefProvider | |
case class SubscribeReceiver(receiverActor: ActorRef) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
find scala/ akka/ spark/ -name '*scala' -type f | xargs -n 1 wc -l >> lines_scala | |
cat lines_scala | awk '{ SUM += $1} END { print SUM }' | |
587927 | |
find hadoop/ -name '*java' -type f | xargs -n 1 wc -l >> lines_java | |
cat lines_java | awk '{ SUM += $1} END { print SUM }' | |
881574 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sealed trait PacketType { def name: String; def id: Int } | |
case object disconnect extends PacketType { val name = "disconnect"; val id = 0 } | |
case object connect extends PacketType { val name = "connect"; val id = 1 } | |
case object heartbeat extends PacketType { val name = "heartbeat"; val id = 2 } | |
case object message extends PacketType { val name = "message"; val id = 3 } | |
case object Json extends PacketType { val name = "json"; val id = 4 } | |
case object event extends PacketType { val name = "event"; val id = 5 } | |
case object ack extends PacketType { val name = "ack"; val id = 6 } | |
case object error extends PacketType { val name = "error"; val id = 7 } | |
case object noop extends PacketType { val name = "noop"; val id = 8 } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-module(erlMessage). | |
-export([start/1,sendMessage/2,msgReceive/0]). | |
% msgReceive blocks on the message queue and waits for a message! | |
% and prints the message as soon as it arrives and again tail recurses! | |
msgReceive() -> | |
receive | |
{Message,Message_from} -> io:format("~p said:~p~n",[Message_from,Message]); | |
Message ->io:format("~p ~n",[Message]) |