Skip to content

Instantly share code, notes, and snippets.

@dakrone
Created May 8, 2010 18:03
Show Gist options
  • Select an option

  • Save dakrone/394685 to your computer and use it in GitHub Desktop.

Select an option

Save dakrone/394685 to your computer and use it in GitHub Desktop.
(ns clomoios.contextsearcher
(:use [clomoios.core])
(:require [opennlp.nlp :as nlp]))
(defprotocol Searcher
"An interface for searching"
(score [this term text] "Score this text in similarity")
(rank [this term text] "Rank sentences in this text"))
(defrecord ContextSearcher [get-sentences tokenize pos-tag])
(extend-protocol Searcher ContextSearcher
(score
[this term text]
(let [get-sentences (:get-sentences this)
tokenizer (:tokenize this)
pos-tagger (:pos-tag this)
score-words (get-scored-terms text term get-sentences tokenizer pos-tagger)]
(score-text text score-words get-sentences tokenizer)))
(rank
[this term text]
(let [get-sentences (:get-sentences this)
tokenizer (:tokenize this)
pos-tagger (:pos-tag this)
score-words (get-scored-terms text term get-sentences tokenizer pos-tagger)]
(reverse (sort-by second (score-sentences text score-words get-sentences tokenizer))))))
(defn make-context-searcher
"Generate a new Context Searcher using the given models. 3 models are
required, a sentence detector model, a tokenizing model and a pos-tagging
model."
[smodel tmodel pmodel]
(let [get-sentences (nlp/make-sentence-detector smodel)
tokenizer (nlp/make-tokenizer tmodel)
pos-tagger (nlp/make-pos-tagger pmodel)]
(ContextSearcher. get-sentences tokenizer pos-tagger)))
; How to refer to the 'score' and 'rank' methods?
;user=> (use 'clomoios.contextsearcher)
;nil
;
;user=> (def foo (make-context-searcher "models/EnglishSD.bin.gz" "models/EnglishTok.bin.gz" "models/tag.bin.gz"))
;#'user/foo
;
;user=> foo
;#:clomoios.contextsearcher.ContextSearcher{:get-sentences #<nlp$make_sentence_detector__248$sentenizer__249 opennlp.nlp$make_sentence_detector__248$sentenizer__249@766b62d1>, :tokenize #<nlp$make_tokenizer__252$tokenizer__253 opennlp.nlp$make_tokenizer__252$tokenizer__253@9c4ff2c>, :pos-tag #<nlp$make_pos_tagger__258$pos_tagger__259 opennlp.nlp$make_pos_tagger__258$pos_tagger__259@64552a2b>}
;
;user=> (score foo "a" "a")
;java.lang.Exception: Unable to resolve symbol: score in this context (NO_SOURCE_FILE:6)
;
;user=> (clomoios.contextsearcher/score foo "a" "a")
;java.lang.Exception: No such var: clomoios.contextsearcher/score (NO_SOURCE_FILE:7)
;
;user=> (clomoios.contextsearcher.ContextSearcher/score foo "a" "a")
;java.lang.IllegalArgumentException: No matching method: score (NO_SOURCE_FILE:8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment