Skip to content

Instantly share code, notes, and snippets.

@language-engineering
Created September 26, 2012 10:17
Show Gist options
  • Select an option

  • Save language-engineering/3787189 to your computer and use it in GitHub Desktop.

Select an option

Save language-engineering/3787189 to your computer and use it in GitHub Desktop.
from nltk.probability import FreqDist
from nltk import Text
#An example list of tokens, replace this list of tokens with one gained from each corpus sample
tokens = ["one","ring","to","rule","them","all"]
#First create a Text object from your sample of tokens
my_text = Text(tokens)
#Next create a FreqDist object from the newly created Text object
my_freqdist = FreqDist(my_text)
#Print items in the FreqDist to show that it worked:
for item,frequency in freqdist.items():
print item,frequency
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment