Skip to content

Instantly share code, notes, and snippets.

@pranjalAI
Created November 28, 2020 18:05
Show Gist options
  • Save pranjalAI/0dcc17ab846c8a942066e7a495230abc to your computer and use it in GitHub Desktop.
Save pranjalAI/0dcc17ab846c8a942066e7a495230abc to your computer and use it in GitHub Desktop.
doc = processArti(content)
doc = tokenize(doc)
doc = [wd for wd in doc if wd not in stop_words]
doc = sorted(set(doc))
doc = " ".join(doc)
doc = tokens(doc)
doc = removeNum(doc)
doc = [i for i in doc if not i==""]
doc = " ".join(doc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment