6. Learning to Classify Text
Detecting patterns is a central part of Natural Language Processing. Words ending in -ed tend to be past tense verbs (5.). Frequent use of will is indicative of news text (3). These observable patterns — word structure and word frequency — happen to correlate with particular aspects of meaning, such as tense and topic. But how did we know where to start looking, which aspects of form to associate with which aspects of meaning?
The goal of this chapter is to answer the following questions:
How can we identify particular features of language data that are salient for classifying it? How can we construct models of language that can be used to perform language processing tasks automatically? What can we learn about language from these models?
http://www.nltk.org/book/ch06.html
Collective intelligence - A book based on python language