Dave Liepmann daveliepmann

Implementing the k-means jump method: Part Four

This fourth and final part of our exploration into the jump method wraps things up by following a basic application of the technique through to a search for results we can be confident in.

We'll reuse functions from rest of the series (Part One, Part Two, Part Three). You'll also want the original paper handy to refer back to their charts.

First, we're going to return to several prior data sets:

Implementing the k-means jump method: Part Three

After implementing Sugar & James jump method and applying it straightforwardly to a few data sets, we're ready to throw it a few curveballs. This will demonstrate both its robustness and some necessary aspects of setting it up for success.

Again we need some functions we built in part one and part two. Feel free to skip this part if you don't need the review and you're not following along with a REPL. If you are following along in the REPL, here's the full Clojure source.

(def iris (i/to-matrix (incd/get-dataset :iris)))

Implementing the k-means jump method: Part Two

After implementing Sugar & James' jump method and exploring its application to Fisher's iris data in Part One of this series, we're now ready to apply the jumps-in-distortions test to some other sample data sets. Pure Clojure source here.

Remember these functions from earlier? We'll be using them again.

(defn assoc-distortions
  "Given a number `transformation-power-y` and a seq of maps

Implementing the k-means jump method: Part One

This paper:

Finding the number of clusters in a data set: An information theoretic approach

CATHERINE A. SUGAR AND GARETH M. JAMES

>Marshall School of Business, University of Southern California

	;; # A whirlwind tour of Maria.cloud

	;; ### 🔺 🔷 🔴 🔶 🔵 🔻
	;; The first thing you should do is draw something:

	(circle 50)

	;; we provide an extensive picture language:

	(layer

	;;;; Fernseheturm

	(def base
	(layer (position 35 90 (colorize "grey" (circle 25)))
	(position 34 0 (colorize "grey" (rectangle 4 300)))
	(position 29 108 (colorize "grey" (rectangle 12 222)))
	(position 29 50 (colorize "grey" (rectangle 12 16)))
	(position 15 315 (colorize "grey" (triangle 40)))
	(position 26 125 (colorize "grey" (rectangle 18 16)))
	(position 24 182 (colorize "grey" (rectangle 5 158)))

	(ns jump-sugar-james.curveball
	(:require [incanter.core :as i]
	[incanter.stats :as incs]
	[incanter.charts :as incc]
	[incanter.datasets :as incd]
	[clj-ml.clusterers :as mlc]
	[clj-ml.data :as mld]))

	;;;; We'll need some functions from earlier.

	(ns jump-sugar-james.application
	(:require [incanter.core :as i]
	[incanter.stats :as incs]
	[incanter.charts :as incc]
	[clj-ml.clusterers :as mlc]
	[clj-ml.data :as mld]
	[clojure.string :as s]))

	;;;; We'll need some functions from earlier.

	(ns jump-sugar-james.irises
	(:require [incanter.core :as i]
	[incanter.stats :as incs]
	[incanter.charts :as incc]
	[incanter.datasets :as incd]
	[clj-ml.clusterers :as mlc]
	[clj-ml.data :as mld]))

	;;;; This paper:
	;; Finding the number of clusters in a data set: An information theoretic approach

	aws lambda invoke \
	--function-name clj-hello \
	--payload \"Dave\" \
	output

	# or...
	aws lambda invoke --function-name clj-hello --payload \"Dave\" output

	# either way, the response from your function goes in the file "output"