Skip to content

Instantly share code, notes, and snippets.

View erasmas's full-sized avatar
🇺🇦

Dmytro Kobza erasmas

🇺🇦
View GitHub Profile
package com.mycompany;
import cascading.flow.Flow;
import cascading.flow.FlowConnector;
import cascading.flow.FlowDef;
import cascading.flow.hadoop2.Hadoop2MR1FlowConnector;
import cascading.pipe.GroupBy;
import cascading.pipe.Pipe;
import cascading.property.AppProps;
import cascading.scheme.Scheme;
@erasmas
erasmas / gist:642f6c6f0f574b68a368
Created March 3, 2015 13:55
Clojure fn that recursively find directories on HDFS containing *.parquet files
(defn dirs-with-parquet
"Recursively find directories on HDFS that contain *.parquet files"
[path]
(let [fs (FileSystem/getLocal (Configuration.))
directory (Path. path)
dir? #(.isDirectory fs %)
contains-parquet (fn [path] (not (empty? (.globStatus fs (Path. path "*.parquet")))))
dirs (tree-seq dir?
(fn [path] (map #(.getPath %) (.listStatus fs path)))
directory)]
@erasmas
erasmas / gist:315eeb0767a18cf3e737
Last active January 10, 2019 06:29
Setting up build status icon using anybar

Following are the steps to setup a fancy build status icon in your OSX bar using AnyBar. At my current project we use Bamboo for CI, which provides REST API to get build statuses that you can visualize in system bar with green or red icon depending on the state of build.

Install Dependencies

# install cask and anybar
brew install caskroom/cask/brew-cask
brew cask install anybar

# install socat and jq
@erasmas
erasmas / fwf.clj
Last active August 29, 2015 14:22
; In R language there’s a very cool function which allows you to read FWF files w/o a hassle:
; http://stat.ethz.ch/R-manual/R-patched/library/utils/html/read.fwf.html
; Following is a Clojure function to parse a line from a FWF file.
(defn fixed-width-format
[s widths]
(->> (reductions (fn [acc n] (+ acc (Math/abs n))) (cons 0 widths))
(partition 2 1)
(map (fn [[start end]] (subs s start end)))
(keep-indexed (fn [idx e] (if (pos? (nth widths idx)) e)))))
(testing
"take-ordered returns the first N elements of an RDD using the natural ordering"
(is (= (-> (s/parallelize c [[1 -1] [2 -2] [3 -3] [4 -4]])
(s/take-ordered 1))
[[1 -1]])))
(testing
"take-ordered returns the first N elements of an RDD as defined by the specified comparator"
(is (= (-> (s/parallelize c [[1 -1] [2 -2] [3 -3] [4 -4]])
(s/take-ordered 1 (comparator (fn [[_ y1] [_ y2]] (< y1 y2)))))

Keybase proof

I hereby claim:

  • I am erasmas on github.
  • I am dmorozov (https://keybase.io/dmorozov) on keybase.
  • I have a public key whose fingerprint is 6200 D027 37D4 E068 BDE5 78AB 186A 11CD 1418 AD13

To claim this, I am signing this object:

(ns hello-world.core
(:require
[cljs.core.async :as async :refer [put! chan <! >! close!]]
[cljs.nodejs :as node])
(:require-macros
[cljs.core.async.macros :refer [go]]))
(enable-console-print!)
(node/enable-util-print!)
#!/usr/bin/env bash
set -e
PN=`basename "$0"`
host=localhost
port=8983
collection=
inputfile=
@erasmas
erasmas / manning-dotd
Last active August 16, 2017 20:09
Simple script to play with Ammonite-REPL
#!/usr/bin/env amm
import $ivy.`org.jsoup:jsoup:1.7.2`
import org.jsoup._
val doc = Jsoup.connect("http://manning.com").get()
val text = doc.select("#dotd-link-non-mobile > em:nth-child(1)").text
println(text)
println("http://manning.com/dotd")