Skip to content

Instantly share code, notes, and snippets.

View cgrand's full-sized avatar

Christophe Grand cgrand

View GitHub Profile
@cgrand
cgrand / retrospec.clj
Created February 15, 2017 15:07
Retro-spec-ing a dataframe
=> (ns user (:require [powderkeg.sql :as sql] [clojure.spec :as s]))
=> (s/def ::age int?)(s/def ::name string?) (s/def ::friends (s/* ::name))
:user/age
:user/name
:user/friends
=> (sql/df [{:name "Gaël" :age 6 :friends ["Alix"]}
{:name "Estelle" :age 4 :friends ["Léana"]}
{:name "Lilly" :age 1 :friends ["Tigger" "Cuddly"]}]
(s/keys :req-un [::name ::age ::friends]))
@cgrand
cgrand / dataframes.clj
Last active February 14, 2017 11:41
DataFrames are coming to Powderkeg!
;; first, define some specs
=> (s/def ::name string?)
(s/def ::age int?)
(s/def ::friends (s/* ::name))
:user/name
:user/age
:user/friends
;; second create a dataframe from a RDD and a spec
=> (sql/df (rdd [{:name "Gaël" :age 6 :friends ["Alix"]}
@cgrand
cgrand / spec.clj
Created February 13, 2017 14:20
from clojure spec to spark schema [PoC]
(ns powderkeg.spec
(:require [clojure.spec :as s])
(:import [org.apache.spark.sql.types DataType DataTypes]))
(defmulti expr first)
(def ^:private regexp-repeat
(s/and
(s/cat :tag any? :type ::datatype)
(s/conformer
=> (class #(* 10 %))
user$eval19119$fn__19120
=> (class #(* 10 %))
user$eval19124$fn__19125
=> (class #(* 10 %))
user$eval19129$fn__19130
=> (class #(* 10 %))
user$eval19134$fn__19135
=> (class #(* 10 %))
user$eval19139$fn__19140
;; simple stats computed using xforms transducers
=> (require '[net.cgrand.xforms :as x])
=> (into {}
(x/by-key #(> % 0.5)
(x/transjuxt {:min x/min :max x/max :avg x/avg :sd x/sd}))
(repeatedly 1e6 rand))
{false
{:min 1.0036018363024368E-6,
:max 0.4999987525112054,
:avg 0.2502046186242099,
@cgrand
cgrand / core.cljc
Last active October 14, 2020 12:18
Mixing macros and code in cljc and supporting clj, cljs and self-hosted cljs, see https://github.com/cgrand/macrovich
;; SEE: https://github.com/cgrand/macrovich
;; macros and code in a single cljc working across clj, cljs and self-hosted cljs
;; require clojurescript from master
(ns foo.core
#?(:cljs (:require-macros
[net.cgrand.meta-macros :refer [macros no-macros]]
[foo.core :refer [add]])
:clj (:require
;; breaking into separate file because inputs are too big
(defn checksum [name]
(apply str (take 5 (map first (sort (fn [[l n] [l' n']] (or (> n n') (and (= n n') (< (int l) (int l'))))) (-> name frequencies (dissoc \-)))))))
(defn day4-1 [input]
(->> (re-seq #"([-a-z]+)-(\d+)\[([a-z]+)\]" input)
(filter (fn [[_ name sector-id sum]] (= (checksum name) sum)))
(map (fn [[_ name sector-id sum]] (Long/parseLong sector-id)))
(reduce +)))
(require '[net.cgrand.xforms :as x])
(defn rollup [dimensions valfn]
(let [[dim & dims] (reverse dimensions)]
(reduce
(fn [xform dim]
(comp
(x/by-key dim xform)
(x/transjuxt
{:detail (x/into {})
(ns advent2016)
;; day1
(def day1-input "R4, R3, R5, L3, L5, R2, L2, R5, L2, R5, R5, R5, R1, R3, L2, L2, L1, R5, L3, R1, L2, R1, L3, L5, L1, R3, L4, R2, R4, L3, L1, R4, L4, R3, L5, L3, R188, R4, L1, R48, L5, R4, R71, R3, L2, R188, L3, R2, L3, R3, L5, L1, R1, L2, L4, L2, R5, L3, R3, R3, R4, L3, L4, R5, L4, L4, R3, R4, L4, R1, L3, L1, L1, R4, R1, L4, R1, L1, L3, R2, L2, R2, L1, R5, R3, R4, L5, R2, R5, L5, R1, R2, L1, L3, R3, R1, R3, L4, R4, L4, L1, R1, L2, L2, L4, R1, L3, R4, L2, R3, L1, L5, R4, R5, R2, R5, R1, R5, R1, R3, L3, L2, L2, L5, R2, L2, R5, R5, L2, R3, L5, R5, L2, R4, R2, L1, R3, L5, R3, R2, R5, L1, R3, L2, R2, R1")
(defn day1-1 [input]
(let [[[x y]] (reduce (fn [[[x y] [dx dy]] [_ dir n]]
(let [n (Long/parseLong n)
[dx dy] (case dir
"R" [dy (- dx)]
(defn mid [a b] (/ (+ a b) 2.0))
(count (take-while #(not= 1.0 %) (iterate #(mid 1 %) 2)))
=> 53
; 53 is not random but tied to the fact that the fraction part of a double is stored on 52 bits