Skip to content

Instantly share code, notes, and snippets.

@ravishchawla
Created June 27, 2018 19:03
Show Gist options
  • Select an option

  • Save ravishchawla/534c4ec722941bfecacd7ae225e98d1c to your computer and use it in GitHub Desktop.

Select an option

Save ravishchawla/534c4ec722941bfecacd7ae225e98d1c to your computer and use it in GitHub Desktop.
# Truncate and Pad reviews at a Maximum cap of 60 words.
max_cap = 60;
X = pad_sequences(reviews_encoded, maxlen=max_cap, truncating='post')
# Obtain a One-hot Y array for each review label.
Y = np.array([[0,1] if '0' in label else [1,0] for label in labels])
# Get a randomized sequence of positions to shuffle reviews
np.random.seed(1024);
random_posits = np.arange(len(X))
np.random.shuffle(random_posits);
# Shuffle X and Y
X = X[random_posits];
Y = Y[random_posits];
# Divide the reviews into Training, Dev, and Test data.
train_cap = int(0.85 * len(X));
dev_cap = int(0.93 * len(X));
X_train, Y_train = X[:train_cap], Y[:train_cap];
X_dev, Y_dev = X[train_cap:dev_cap], Y[train_cap:dev_cap];
X_test, Y_test = X[dev_cap:], Y[dev_cap:]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment