Skip to content

Instantly share code, notes, and snippets.

@robinkraft
Created October 31, 2012 23:08
Show Gist options
  • Select an option

  • Save robinkraft/3990535 to your computer and use it in GitHub Desktop.

Select an option

Save robinkraft/3990535 to your computer and use it in GitHub Desktop.
convert raw rain timeseries from pail to stretched version in sequence file
(use 'forma.hadoop.jobs.scatter)
(require '[forma.source.rain :as rain])
(require '[forma.utils :as u])
(let [nodata -9999.0
src (pail-tap "/mnt/hgfs/robin/delete/timeseries/" "precl" "500" "32")
out-loc (hfs-seqfile "/mnt/hgfs/robin/delete/rain-redo")]
(<- [?s-res ?mod-h ?mod-v ?sample ?line ?new-start ?new-rain]
(src _ ?dc)
(thrift/unpack ?dc :> _ ?loc ?data ?t-res _)
(thrift/unpack ?loc :> ?s-res ?mod-h ?mod-v ?sample ?line)
(thrift/get-field-value ?data :> ?ts-obj)
(thrift/unpack ?ts-obj :> ?start _ ?ts)
(thrift/unpack* ?ts :> ?rain)
(u/replace-from-left* nodata ?rain :default 0 :> ?clean-rain)
(rain/stretch-rain ?t-res "16" ?start ?clean-rain :> ?new-start ?new-rain)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment