as_gated

tfsnippet.layers.as_gated(layer_fn, sigmoid_bias=2.0, default_name=None)

Wrap a layer function into a gated layer function.

For example, the following gated_dense function:

@add_arg_scope
def gated_dense(inputs, units, activation_fn=None, sigmoid_bias=2.,
                name=None, scope=None, **kwargs):
    with tf.name_scope(scope, default_name=name):
        gate = tf.sigmoid(sigmoid_bias +
            dense(inputs, units, scope='gate', **kwargs))
        return gate * dense(
            inputs, units, activation_fn=activation_fn, scope='main',
            **kwargs
        )

can be deduced by applying this function:

gated_dense = as_gated(dense)

Parameters:

• layer_fn – The layer function to be wrapped.
• sigmoid_bias – The constant bias added to the gate before applying the sigmoid activation.
• default_name – Default name of variable scope.

Returns:

The wrapped gated layer function.

Notes

If a layer supports gated argument (e.g., spt.layers.dense()), it is generally better to use that argument, instead of using this as_gated() wrapper on the layer.