This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # create bag-of-words | |
| all_words = [] | |
| for message in sms_data: | |
| words = word_tokenize(message) | |
| for w in words: | |
| all_words.append(w) | |
| all_words = nltk.FreqDist(all_words) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| encoder = LabelEncoder() | |
| Y = encoder.fit_transform(classes) | |
| # Now lets do it for all the messages | |
| messages = list(zip(sms_data, Y)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # define a seed for reproducibility | |
| seed = 1 | |
| np.random.seed = seed | |
| np.random.shuffle(messages) | |
| # call find_features function for each SMS message | |
| featuresets = [(find_features(text), label) for (text, label) in messages] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # we can split the featuresets into training and testing datasets using sklearn | |
| from sklearn import model_selection | |
| # split the data into training and testing datasets | |
| training = featuresets | |
| # Define models to train | |
| names = ["Logistic Regression"] | |
| classifiers = [ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| for name, model in models: | |
| nltk_model = SklearnClassifier(model) | |
| classifier = nltk_model.train(training) | |
| f = open(name + ' Classifier.pickle', 'wb') | |
| pickle.dump(classifier, f) | |
| f.close |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| classifier_s = open("sms_classifier_pickle/" + name + ' Classifier.pickle', "rb") | |
| sms_classifier = pickle.load(classifier_s) | |
| classifier_s.close() | |
| result = sms_classifier.classify(find_features(preproccess_text('ENTER MESSAGE TO CLASSIFY HERE'))) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import os | |
| import re | |
| import sys | |
| import nltk | |
| import random | |
| import pickle | |
| import numpy as np | |
| import pandas as pd | |
| from sklearn.svm import SVC | |
| from sklearn.naive_bayes import MultinomialNB |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from flask import Flask | |
| app = Flask(__name__) | |
| if __name__ == "__main__": | |
| app.run(host='0.0.0.0') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import os | |
| from faunadb import query as q | |
| from faunadb.objects import Ref | |
| from faunadb.client import FaunaClient | |
| def get(index, data): | |
| try: | |
| serverClient = FaunaClient(secret=os.environ.get("FAUNA_SERVER_SECRET")) | |
| res = serverClient.query(q.get(q.match(q.index(index), data))) | |
| res["data"]["ref_id"] = res["ref"].id() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def get_multiple(index, data=None): | |
| try: | |
| serverClient = FaunaClient(secret=os.environ.get("FAUNA_SERVER_SECRET")) | |
| res_arr = [] | |
| if data is None: | |
| res = serverClient.query( | |
| q.map_( | |
| q.lambda_("data", q.get(q.var("data"))), | |
| q.paginate(q.match(q.index(index))) | |
| ) |