A keras bug reproduce.

KerasBug.md

The script shows a wierd bug using keras with tensorflow.

When you use the tf.while_loop function

1. if you include the Dropout layer or set the total number of epochs to greater than 10, then the program hangs and never returns.
2. Otherwise, it works fine.

I've updated to the latest version of keras and tensorflow. This bug persists. It seems to be a bug with keras. Since I could not reproduce this with pure tensorflow code.

Reproduce.py

import os
# supress tensorflow logging other than errors
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

mport numpy as np
import tensorflow as tf

import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Dropout

print(keras.__version__)
print(tf.__version__)

sess = tf.InteractiveSession()
K.set_session(sess)


model = Sequential([
    Dense(10, input_shape=[3,]),
    Dropout(0.5),               # comment out this line and it works
    Dense(1)])
model.compile(optimizer='adam', loss='binary_crossentropy',
              metrics=['accuracy'])


nb_epoch = tf.constant(20)

def _cond(xx, i):
    return i < nb_epoch

def _body(xx, i):
    y = model(xx)
    dy_dx, = tf.gradients(y, xx)
    xx = tf.stop_gradient(xx + tf.sign(dy_dx))
    i += 1
    return xx, i

x = tf.placeholder(tf.float32, (None, 3))

xx = tf.identity(x)
i = tf.Variable(0)
xx, i = tf.while_loop(_cond, _body, [xx, i])

init = tf.global_variables_initializer()
sess.run(init)

val = sess.run(xx, feed_dict={
    x: np.random.random((1, 3)),
    K.learning_phase(): 0
})
print(val)