Tobias Bertelsen tbertelsen

tbertelsen / pearson.scala

Last active February 28, 2021 22:14 — forked from kaja47/pearson.scala

Calculating pearson for Breeze vectors

	import breeze.linalg._
	import breeze.stats._
	import scala.math.sqrt


	/**
	* Effecient for sparse vectors. Scales in O(activeSize)
	*/
	// Must take SparseVector, for implicits to be linked correctly
	def pearson(a: SparseVector[Double], b: SparseVector[Double]): Double = {

tbertelsen / gist:446b9062e645cbeaf646

Created February 4, 2015 10:38

Script for testing precedence of Spark configuration

	#!/bin/bash

	echo "SPARK_WORKER_DIR=/tmp/sparktest/file/" > $SPARK_HOME/conf/spark-env.sh
	#echo "" > $SPARK_HOME/conf/spark-env.sh

	LOCAL_HOSTNAME=`hostname`

	$SPARK_HOME/sbin/start-master.sh
	echo "sleeping for 10 s"
	sleep 10