Since Keras 2.0 removed the Highway Network layer, here's my attempt at implementing something equivalent using the functional API

keras2-highway-network.py

import keras.backend as K
from keras.layers import Dense, Activation, Multiply, Add, Lambda
import keras.initializers

def highway_layers(value, n_layers, activation="tanh", gate_bias=-3):
    dim = K.int_shape(value)[-1]
    gate_bias_initializer = keras.initializers.Constant(gate_bias)
    for i in range(n_layers):     
        gate = Dense(units=dim, bias_initializer=gate_bias_initializer)(value)
        gate = Activation("sigmoid")(gate)
        negated_gate = Lambda(
            lambda x: 1.0 - x,
            output_shape=(dim,))(gate)
        transformed = Dense(units=dim)(value)
        transformed = Activation(activation)(value)
        transformed_gated = Multiply()([gate, transformed])
        identity_gated = Multiply()([negated_gate, value])
        value = Add()([transformed_gated, identity_gated])
    return value

Thanks for the implementation. What is the utility of the "transformed" variable in line 14? As right in the next line you are defining a new expression for "transformed" which only uses the "value" variable.

I believe line 15 should be : transformed = Activation(activation)(transformed)

Hey, how could you add this layer into a CNN using Sequential in Keras?