bcambel

# Required downloads:
sudo wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64/cuda-repo-ubuntu1404_8.0.44-1_amd64.deb
# NVIDIA-Linux-x86_64-367.27.run
# cuda_8.0.27_linux.run
# cudnn-8.0-linux-x64-v5.0-ga.tgz

sudo apt-get install build-essential
sudo apt-get install linux-image-extra-`uname -r`
sudo ./NVIDIA-Linux-x86_64-367.27.run
./cuda_8.0.27_linux.run --extract=`pwd`/extracts

bcambel

(with-open [file-stream (FileInputStream. ten-gb-filename)]
  (let [channel (chan 500)
        ;; make four workers to read byte arrays off the channel:
        counters (for [_ (range 4)] 
                   (go-loop [newline-count 0]
                     (let [barray (async/<! channel)]
                       (if (nil? barray) ;; channel is closed
                         newline-count
                         (recur (+ newline-count
                                   (count-newlines barray)))))))]

bcambel

-- show running queries (pre 9.2)
SELECT procpid, age(query_start, clock_timestamp()), usename, current_query 
FROM pg_stat_activity 
WHERE current_query != '<IDLE>' AND current_query NOT ILIKE '%pg_stat_activity%' 
ORDER BY query_start desc;

-- show running queries (9.2)
SELECT pid, age(query_start, clock_timestamp()), usename, query 
FROM pg_stat_activity 
WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%' 

bcambel

(ns mqtt-insertion.core
  (:require [clojure.core.async :refer :all])
  (:import (org.eclipse.paho.client.mqttv3
            MqttCallback
            MqttAsyncClient
            MqttConnectOptions
            MqttDeliveryToken
            MqttException
            MqttMessage
            MqttTopic

bcambel

; it's a bit cumbersome to set up and there's the unfortunate need to ignore the tag
; in the individual methods, but this allows you to leave the interpretation of open 
; variants, well, *open* for extension by multimethod.

; dispatch off the first argument, which will be the tag
(defmethod command-multi (fn [tag & data] tag))

; the first argument to the *method* is still the tag
(defmulti command-multi :print [_ val] (println val))
(defmulti command-multi :read [_ fname] (slurp fname))

bcambel

The Anatomy of an Onyx Program

In this tutorial, we'll take an in-depth view of what's happening when you execute a simple Onyx program. All of the code can be found in the Onyx Starter repository if you'd like to follow along. The code uses the development environment with HornetQ and ZooKeeper running in memory, so you don't need additional dependencies to run the example for yourself on your machine.

The Workflow

At the core of the program is the workflow - the flow of data that we ingest, apply transformations to, and send to an output for storage. In this program, we're going to ingest some sentences from an input source, split the sentence into individual words, play with capitalization, and add a suffix. Finally, we'll send the transformed data to an output source.

Let's examine the workflow pictorially:

bcambel

(ns rbl.feature-extraction.user-agent
  (:use [clojure.core.memoize :only [memo]])
  (:require [clojure.tools.logging :as log])
  (:import [nl.bitwalker.useragentutils UserAgent DeviceType Browser OperatingSystem]))

(defn str->features [string]
  (try
    (let [user-agent (UserAgent. (or string ""))]
      {:browser_group (-> user-agent .getBrowser .getGroup .getName)
       :os_group (-> user-agent .getOperatingSystem .getGroup .getName)

bcambel

(ns aud_cas.print_fields
  (:use cascalog.api)
  (:require [clojure.string :as str])
  )

(defn parse_input_record
  "Parse the text of the input record into fields in a map"
  [input_record]
  (let [prefix_string (get (str/split input_record #"\: ") 0)
        prefix_pairs (str/split prefix_string #" ")

bcambel

Clojure Destructuring Tutorial and Cheat Sheet

(Related blog post)

Simply put, destructuring in Clojure is a way extract values from a datastructure and bind them to symbols, without having to explicitly traverse the datstructure. It allows for elegant and concise Clojure code.

Vectors

bcambel

(ns example.errors)

(defn clean-address [params]
  "Ensure (params :address) is present"
  (if (empty? (params :address))
    [nil "Please enter your address"]
    [params nil]))

(defn clean-email [params]
  "Ensure (params :email) matches *@*.*"