Skip to content

Instantly share code, notes, and snippets.

@chulman444
Created November 27, 2017 16:02
Show Gist options
  • Save chulman444/e9021c45750c8fc4018d41ac7e836a95 to your computer and use it in GitHub Desktop.
Save chulman444/e9021c45750c8fc4018d41ac7e836a95 to your computer and use it in GitHub Desktop.
Code typed from the [video](https://youtu.be/UeheTiBJ0Io?t=16m46s)
video = tf.keras.layers.Input(shape=(None, 150, 150, 3))
cnn = tf.keras.applications.InceptionV3(
weights='imagenet',
include_top = False,
pool='avg')
cnn.trainable = False
encoded_frames = tf.keras.layers.TimeDistributed(cnn)(video)
encoded_vid = tf.layers.LSTM(256)(encoded_frames)
question = tf.keras.layers.Input(shape=(100), dtype='int32')
x = tf.keras.layers.Embedding(10000, 256, mask_zero=True)(question)
encoded_q = tf.keras.layers.LSTM(128)(x)
x = tf.keras.layers.concat([encoded_vid, encoded_q])
x = tf.keras.layers.Dense(128, activation=tf.nn.relu)(x)
outputs = tf.keras.layers.Dense(1000)(x)
model = tf.keras.models.Model([video, question], outputs)
model.compile(
optimizer=tf.AdamOptimizer(),
loss=tf.softmax_crossentropy_with_logits)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment