Skip to content

Instantly share code, notes, and snippets.

@eileen-code4fun
Last active January 17, 2022 20:50
Show Gist options
  • Save eileen-code4fun/886999b3aaf5caa5ae4613bfb07b7f1e to your computer and use it in GitHub Desktop.
Save eileen-code4fun/886999b3aaf5caa5ae4613bfb07b7f1e to your computer and use it in GitHub Desktop.
import re
def lower_remove_punctuation(txt):
txt = txt.lower()
return re.sub(r'[^\w\s]','',txt)
train = preprocess(train_dataset, lower_remove_punctuation)
test = preprocess(test_dataset, lower_remove_punctuation)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment