Skip to content

Instantly share code, notes, and snippets.

@jeroenvandijk

jeroenvandijk/project.clj

Created December 8, 2011 16:40
Show Gist options
  • Save jeroenvandijk/1447540 to your computer and use it in GitHub Desktop.
Save jeroenvandijk/1447540 to your computer and use it in GitHub Desktop.
Download ZIP
Baisc scraper with Aleph and Enlive
Raw
project.clj
(defproject scraper "1.0.0-SNAPSHOT"
:dependencies [
[org.clojure/clojure "1.2.1"]
[aleph "0.2.0"]
[enlive "1.0.0-SNAPSHOT"]
])
Raw
simple_scraper.clj
(require 'lamina.core 'aleph.http 'aleph.formats 'net.cgrand.enlive-html)
(defn handler [resp]
(let [body-channel (lamina.core/map* aleph.formats/bytes->input-stream(:body resp))]
(lamina.core/receive body-channel #(println (net.cgrand.enlive-html/select (net.cgrand.enlive-html/html-resource %) [:a])))))
(lamina.core/async
(handler (aleph.http/http-request {:method :get :url "http://slashdot.org"})))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment