tfsnippet.flows

class tfsnippet.flows.Flow(dtype=tf.float32, name=None, scope=None)

Bases: tfsnippet.utils.scope.VarScopeObject

The basic class for normalizing flows.

A normalizing flow transforms a random variable x into y by an (implicitly) invertible mapping \(y = f(x)\), whose Jaccobian matrix determinant \(\det \frac{\partial f(x)}{\partial x} \neq 0\), thus can derive \(\log p(y)\) from given \(\log p(x)\).

__init__(dtype=tf.float32, name=None, scope=None)

Construct a new Flow.

Parameters:
dtype

The data type of y.

Returns:The data type of y.
Return type:tf.DType
explicitly_invertible

Whether or not this flow is explicitly invertible?

If a flow is not explicitly invertible, then it only supports to transform x into y, and corresponding \(\log p(x)\) into \(\log p(y)\). It cannot compute \(\log p(y)\) directly without knowing x, nor can it transform x back into y.

Returns:
A boolean indicating whether or not the flow is explicitly
invertible.
Return type:bool
inverse_transform(y, compute_x=True, compute_log_det=True, name=None)

Transform y into x, and the log-determinant of f^{-1} at y, i.e., \(\log \det \frac{\partial f^{-1}(y)}{\partial y}\).

Parameters:
  • y (Tensor) – The samples of y.
  • compute_x (bool) – Whether or not to compute \(x = f^{-1}(y)\)? Default True.
  • compute_log_det (bool) – Whether or not to compute the log-determinant? Default True.
  • name (str) – If specified, will use this name as the TensorFlow operational name scope.
Returns:

x and the log-determinant.

The items in the returned tuple might be None if corresponding compute_? argument is set to False.
Return type:

(tf.Tensor, tf.Tensor)
Raises:
transform(x, compute_y=True, compute_log_det=True, name=None)

Transform x into y, and the log-determinant of f at x, i.e., \(\log \det \frac{\partial f(x)}{\partial x}\).

Parameters:
  • x (Tensor) – The samples of x.
  • compute_y (bool) – Whether or not to compute \(y = f(x)\)? Default True.
  • compute_log_det (bool) – Whether or not to compute the log-determinant? Default True.
  • name (str) – If specified, will use this name as the TensorFlow operational name scope.
Returns:

y and the log-determinant.

The items in the returned tuple might be None if corresponding compute_? argument is set to False.
Return type:

(tf.Tensor, tf.Tensor)
Raises:

RuntimeError – If both compute_y and compute_log_det are set to False.
class tfsnippet.flows.MultiLayerFlow(n_layers, dtype=tf.float32, name=None, scope=None)

Bases: tfsnippet.flows.base.Flow

Base class for multi-layer normalizing flows.

__init__(n_layers, dtype=tf.float32, name=None, scope=None)

Construct a new MultiLayerFlow.

Parameters:
  • n_layers (int) – Number of flow layers.
  • dtype – The data type of the transformed y.
  • name (str) – Optional name of this VariableSaver (argument of VarScopeObject).
  • scope (str) – Optional scope of this VariableSaver (argument of VarScopeObject).
_create_layer_params(layer_id)

Create layer

Parameters:layer_id (int) – The integer ID of the layer.
Returns:The layer parameters.
Return type:dict[str, tf.Variable or tf.Tensor]
get_layer_params(layer_id, names)

Get layer parameters.

Parameters:
  • layer_id (int) – The integer ID of the layer.
  • names (Iterable[str]) – The names of the parameters to get.
Returns:

The layer parameters.
Return type:

list[tf.Variable or tf.Tensor]
n_layers

Get the number of flow layers.

Returns:The number of flow layers.
Return type:int
class tfsnippet.flows.FlowDistribution(distribution, flow)

Bases: tfsnippet.distributions.base.Distribution

Transform a Distribution by a Flow, as a new distribution.

__init__(distribution, flow)

Construct a new FlowDistribution from the given distribution.

Parameters:
  • distribution (Distribution) – The distribution to transform from. It must be continuous,
  • flow (Flow) – A normalizing flow to transform the distribution.
batch_shape

Get the batch shape of the samples.

Returns:The batch shape as tensor.
Return type:tf.Tensor
distribution

Get the base distribution.

Returns:The base distribution to transform from.
Return type:Distribution
dtype

Get the data type of samples.

Returns:Data type of the samples.
Return type:tf.DType
flow

Get the transformation flow.

Returns:The transformation flow.
Return type:Flow
get_batch_shape()

Get the static batch shape of the samples.

Returns:The batch shape.
Return type:tf.TensorShape
get_value_shape()

Get the static value shape of an individual sample.

Returns:The static value shape.
Return type:tf.TensorShape
is_continuous

Whether or not the distribution is continuous?

Returns:A boolean indicating whether it is continuous.
Return type:bool
is_reparameterized

Whether or not the distribution is re-parameterized?

The re-parameterization trick is proposed in “Auto-Encoding Variational Bayes” (Kingma, D.P. and Welling), allowing the gradients to be propagated back along the samples. Note that the re-parameterization can be disabled by specifying is_reparameterized = False as an argument of sample().

Returns:A boolean indicating whether it is re-parameterized.
Return type:bool
log_prob(given, group_ndims=0, name=None)

Compute the log-densities of x against the distribution.

Parameters:
  • given (Tensor) – The samples to be tested.
  • group_ndims (int or tf.Tensor) – If specified, the last group_ndims dimensions of the log-densities will be summed up. (default 0)
  • name (str) – TensorFlow name scope of the graph nodes. (default “log_prob”).
Returns:

The log-densities of given.
Return type:

tf.Tensor
sample(n_samples=None, group_ndims=0, is_reparameterized=None, compute_density=None, name=None)

Generate samples from the distribution.

Parameters:
  • n_samples (int or tf.Tensor or None) – A 0-D int32 Tensor or None. How many independent samples to draw from the distribution. The samples will have shape [n_samples] + batch_shape + value_shape, or batch_shape + value_shape if n_samples is None.
  • group_ndims (int or tf.Tensor) – Number of dimensions at the end of [n_samples] + batch_shape to be considered as events group. This will effect the behavior of log_prob() and prob(). (default 0)
  • is_reparameterized (bool) – If True, raises RuntimeError if the distribution is not re-parameterized. If False, disable re-parameterization even if the distribution is re-parameterized. (default None, following the setting of distribution)
  • compute_density (bool) – Whether or not to immediately compute the log-density for the samples? (default None, determine by the distribution class itself)
  • name (str) – TensorFlow name scope of the graph nodes. (default “sample”).
Returns:

The samples as

StochasticTensor.
Return type:

tfsnippet.stochastic.StochasticTensor
value_shape

Get the value shape of an individual sample.

Returns:The value shape as tensor.
Return type:tf.Tensor
class tfsnippet.flows.PlanarNormalizingFlow(n_units, n_layers=1, w_initializer=<tensorflow.python.ops.init_ops.RandomNormal object>, b_initializer=<tensorflow.python.ops.init_ops.Zeros object>, u_initializer=<tensorflow.python.ops.init_ops.RandomNormal object>, w_regularizer=None, b_regularizer=None, u_regularizer=None, trainable=True, dtype=tf.float32, name=None, scope=None)

Bases: tfsnippet.flows.base.MultiLayerFlow

Planar Normalizing Flow with activation function tanh as well as the invertibility trick from (Danilo 2016). x is assumed to be a 1-D random variable.

\[\begin{split}\begin{aligned} \mathbf{y} &= \mathbf{x} + \mathbf{\hat{u}} \tanh(\mathbf{w}^\top\mathbf{x} + b) \\ \mathbf{\hat{u}} &= \mathbf{u} + \left[m(\mathbf{w}^\top \mathbf{u}) - (\mathbf{w}^\top \mathbf{u})\right] \cdot \frac{\mathbf{w}}{\|\mathbf{w}\|_2^2} \\ m(a) &= -1 + \log(1+\exp(a)) \end{aligned}\end{split}\]
__init__(n_units, n_layers=1, w_initializer=<tensorflow.python.ops.init_ops.RandomNormal object>, b_initializer=<tensorflow.python.ops.init_ops.Zeros object>, u_initializer=<tensorflow.python.ops.init_ops.RandomNormal object>, w_regularizer=None, b_regularizer=None, u_regularizer=None, trainable=True, dtype=tf.float32, name=None, scope=None)

Construct a new PlanarNormalizingFlow.

Parameters:
  • n_units (int) – The size of the last axis of x.
  • n_layers (int) – The number of normalizing flow layers. (default 1)
  • w_initializer – The initializer for parameter w.
  • b_initializer – The initializer for parameter b.
  • u_initializer – The initializer for parameter u.
  • w_regularizer – The regularizer for parameter w, optional.
  • b_regularizer – The regularizer for parameter b, optional.
  • u_regularizer – The regularizer for parameter u, optional.
  • trainable (bool) – Whether or not the parameters are trainable? (default True)
  • dtype – The data type of the transformed y.
  • name (str) – Optional name of this Flow (argument of VarScopeObject).
  • scope (str) – Optional scope of this Flow (argument of VarScopeObject).
_create_layer_params(layer_id)

Create layer

Parameters:layer_id (int) – The integer ID of the layer.
Returns:The layer parameters.
Return type:dict[str, tf.Variable or tf.Tensor]
explicitly_invertible

Whether or not this flow is explicitly invertible?

If a flow is not explicitly invertible, then it only supports to transform x into y, and corresponding \(\log p(x)\) into \(\log p(y)\). It cannot compute \(\log p(y)\) directly without knowing x, nor can it transform x back into y.

Returns:
A boolean indicating whether or not the flow is explicitly
invertible.
Return type:bool
n_units

Get the size of the last axis of x.

Returns:The size of the last axis of x.
Return type:int
class tfsnippet.flows.SequentialFlow(flows, name=None, scope=None)

Bases: tfsnippet.flows.base.MultiLayerFlow

Manage a sequential list of Flow instances.

__init__(flows, name=None, scope=None)

Construct a new SequentialFlow.

Parameters:flows (Iterable[Flow]) – The flow list.
_create_layer_params(layer_id)

Create layer

Parameters:layer_id (int) – The integer ID of the layer.
Returns:The layer parameters.
Return type:dict[str, tf.Variable or tf.Tensor]
explicitly_invertible

Whether or not this flow is explicitly invertible?

If a flow is not explicitly invertible, then it only supports to transform x into y, and corresponding \(\log p(x)\) into \(\log p(y)\). It cannot compute \(\log p(y)\) directly without knowing x, nor can it transform x back into y.

Returns:
A boolean indicating whether or not the flow is explicitly
invertible.
Return type:bool
flows

Get the immutable flow list.

Returns:The immutable flow list.
Return type:tuple[Flow]