Skip to content

Instantly share code, notes, and snippets.

@language-engineering
Created October 9, 2012 14:52
Show Gist options
  • Select an option

  • Save language-engineering/3859317 to your computer and use it in GitHub Desktop.

Select an option

Save language-engineering/3859317 to your computer and use it in GitHub Desktop.
import nltk
from sussex_nltk.corpus_readers import TwitterCorpusReader
tcr = TwitterCorpusReader()
tokens = tcr.sample_words_by_sents(25000) #get a sample of tokens
fd = nltk.probability.FreqDist(tokens) #build a frequency distribution over tokens
probability_distribution = nltk.probability.LidstoneProbDist(fd, 0.001) #build a probability distribution
#Create a spell checker with new probability distribution
s = SpellChecker(probability_distribution)
#Now use the spellchecker how you normally would
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment