Distribution

class tfsnippet.Distribution

Bases: object

Base class for probability distributions.

A Distribution object receives inputs as distribution parameters, generating samples and computing densities according to these inputs. The shape of the inputs can have more dimensions than the nature shape of the distribution parameters, since Distribution is designed to work with batch parameters, samples and densities.

The shape of the parameters of a Distribution object would be decomposed into batch_shape + param_shape, with param_shape being the nature shape of the parameter. For example, a 5-class Categorical distribution with class probabilities of shape (3, 4, 5) would have (3, 4) as the batch_shape, with (5,) as the param_shape, corresponding to the probabilities of 5 classes.

Generating n samples from a Distribution object would result in tensors with shape [n] (sample_shape) + batch_shape + value_shape, with value_shape being the nature shape of an individual sample from the distribution. For example, the value_shape of a Categorical is (), such that the sample shape would be (3, 4), provided the shape of class probabilities is (3, 4, 5).

Computing the densities (i.e., prob(x) or log_prob(x)) of samples involves broadcasting these samples against the distribution parameters. These samples should be broadcastable against batch_shape + value_shape. Suppose the shape of the samples can be decomposed into sample_shape + batch_shape + value_shape, then by default, the shape of the densities should be sample_shape + batch_shape, i.e., each individual sample resulting in an individual density value.

Attributes Summary

batch_shape Get the batch shape of the samples.
dtype Get the data type of samples.
is_continuous Whether or not the distribution is continuous?
is_reparameterized Whether or not the distribution is re-parameterized?
value_shape Get the value shape of an individual sample.

Methods Summary

get_batch_shape() Get the static batch shape of the samples.
get_value_shape() Get the static value shape of an individual sample.
log_prob(given[, group_ndims, name]) Compute the log-densities of x against the distribution.
prob(given[, group_ndims, name]) Compute the densities of x against the distribution.
sample([n_samples, group_ndims, …]) Generate samples from the distribution.

Attributes Documentation

batch_shape

Get the batch shape of the samples.

Returns:The batch shape as tensor.
Return type:tf.Tensor
dtype

Get the data type of samples.

Returns:Data type of the samples.
Return type:tf.DType
is_continuous

Whether or not the distribution is continuous?

Returns:A boolean indicating whether it is continuous.
Return type:bool
is_reparameterized

Whether or not the distribution is re-parameterized?

The re-parameterization trick is proposed in “Auto-Encoding Variational Bayes” (Kingma, D.P. and Welling), allowing the gradients to be propagated back along the samples. Note that the re-parameterization can be disabled by specifying is_reparameterized = False as an argument of sample().

Returns:A boolean indicating whether it is re-parameterized.
Return type:bool
value_shape

Get the value shape of an individual sample.

Returns:The value shape as tensor.
Return type:tf.Tensor

Methods Documentation

get_batch_shape()

Get the static batch shape of the samples.

Returns:The batch shape.
Return type:tf.TensorShape
get_value_shape()

Get the static value shape of an individual sample.

Returns:The static value shape.
Return type:tf.TensorShape
log_prob(given, group_ndims=0, name=None)

Compute the log-densities of x against the distribution.

Parameters:
  • given (Tensor) – The samples to be tested.
  • group_ndims (int or tf.Tensor) – If specified, the last group_ndims dimensions of the log-densities will be summed up. (default 0)
  • name – TensorFlow name scope of the graph nodes. (default “log_prob”).
Returns:

The log-densities of given.
Return type:

tf.Tensor
prob(given, group_ndims=0, name=None)

Compute the densities of x against the distribution.

Parameters:
  • given (Tensor) – The samples to be tested.
  • group_ndims (int or tf.Tensor) – If specified, the last group_ndims dimensions of the log-densities will be summed up. (default 0)
  • name – TensorFlow name scope of the graph nodes. (default “prob”).
Returns:

The densities of given.
Return type:

tf.Tensor
sample(n_samples=None, group_ndims=0, is_reparameterized=None, compute_density=None, name=None)

Generate samples from the distribution.

Parameters:
  • n_samples (int or tf.Tensor or None) – A 0-D int32 Tensor or None. How many independent samples to draw from the distribution. The samples will have shape [n_samples] + batch_shape + value_shape, or batch_shape + value_shape if n_samples is None.
  • group_ndims (int or tf.Tensor) – Number of dimensions at the end of [n_samples] + batch_shape to be considered as events group. This will effect the behavior of log_prob() and prob(). (default 0)
  • is_reparameterized (bool) – If True, raises RuntimeError if the distribution is not re-parameterized. If False, disable re-parameterization even if the distribution is re-parameterized. (default None, following the setting of distribution)
  • compute_density (bool) – Whether or not to immediately compute the log-density for the samples? (default None, determine by the distribution class itself)
  • name – TensorFlow name scope of the graph nodes. (default “sample”).
Returns:

The samples as

StochasticTensor.
Return type:

tfsnippet.stochastic.StochasticTensor