Created
June 27, 2018 19:03
-
-
Save ravishchawla/534c4ec722941bfecacd7ae225e98d1c to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Truncate and Pad reviews at a Maximum cap of 60 words. | |
| max_cap = 60; | |
| X = pad_sequences(reviews_encoded, maxlen=max_cap, truncating='post') | |
| # Obtain a One-hot Y array for each review label. | |
| Y = np.array([[0,1] if '0' in label else [1,0] for label in labels]) | |
| # Get a randomized sequence of positions to shuffle reviews | |
| np.random.seed(1024); | |
| random_posits = np.arange(len(X)) | |
| np.random.shuffle(random_posits); | |
| # Shuffle X and Y | |
| X = X[random_posits]; | |
| Y = Y[random_posits]; | |
| # Divide the reviews into Training, Dev, and Test data. | |
| train_cap = int(0.85 * len(X)); | |
| dev_cap = int(0.93 * len(X)); | |
| X_train, Y_train = X[:train_cap], Y[:train_cap]; | |
| X_dev, Y_dev = X[train_cap:dev_cap], Y[train_cap:dev_cap]; | |
| X_test, Y_test = X[dev_cap:], Y[dev_cap:] |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment