Distribution¶

class tfsnippet.Distribution(dtype, is_continuous, is_reparameterized, batch_shape, batch_static_shape, value_ndims)¶

Bases: object

Base class for probability distributions.

A Distribution object receives inputs as distribution parameters, generating samples and computing densities according to these inputs. The shape of the inputs can have more dimensions than the nature shape of the distribution parameters, since Distribution is designed to work with batch parameters, samples and densities.

The shape of the parameters of a Distribution object would be decomposed into batch_shape + param_shape, with param_shape being the nature shape of the parameter. For example, a 5-class Categorical distribution with class probabilities of shape (3, 4, 5) would have (3, 4) as the batch_shape, with (5,) as the param_shape, corresponding to the probabilities of 5 classes.

Generating n samples from a Distribution object would result in tensors with shape [n] (sample_shape) + batch_shape + value_shape, with value_shape being the nature shape of an individual sample from the distribution. For example, the value_shape of a Categorical is (), such that the sample shape would be (3, 4), provided the shape of class probabilities is (3, 4, 5).

Computing the densities (i.e., prob(x) or log_prob(x)) of samples involves broadcasting these samples against the distribution parameters. These samples should be broadcastable against batch_shape + value_shape. Suppose the shape of the samples can be decomposed into sample_shape + batch_shape + value_shape, then by default, the shape of the densities should be sample_shape + batch_shape, i.e., each individual sample resulting in an individual density value.

Attributes Summary

`base_distribution`	Get the base distribution of this distribution.
`batch_shape`	Get the batch shape of the samples.
`dtype`	Get the data type of samples.
`is_continuous`	Whether or not the distribution is continuous?
`is_reparameterized`	Whether or not the distribution is re-parameterized?
`value_ndims`	Get the number of value dimensions in samples.

Methods Summary

`batch_ndims_to_value`(ndims)	Convert the last few batch_ndims into value_ndims.
`expand_value_ndims`(ndims)	Convert the last few batch_ndims into value_ndims.
`get_batch_shape`()	Get the static batch shape of the samples.
`log_prob`(given[, group_ndims, name])	Compute the log-densities of x against the distribution.
`prob`(given[, group_ndims, name])	Compute the densities of x against the distribution.
`sample`([n_samples, group_ndims, …])	Generate samples from the distribution.

Attributes Documentation

base_distribution¶

Get the base distribution of this distribution.

For distribution other than tfsnippet.BatchToValueDistribution, this property should return this distribution itself.

Returns:	The base distribution.
Return type:	Distribution

batch_shape¶

Get the batch shape of the samples.

Returns:	The batch shape as tensor.
Return type:	tf.Tensor

dtype¶

Get the data type of samples.

Returns:	Data type of the samples.
Return type:	tf.DType

is_continuous¶

Whether or not the distribution is continuous?

Returns:	A boolean indicating whether it is continuous.
Return type:	bool

is_reparameterized¶

Whether or not the distribution is re-parameterized?

The re-parameterization trick is proposed in “Auto-Encoding Variational Bayes” (Kingma, D.P. and Welling), allowing the gradients to be propagated back along the samples. Note that the re-parameterization can be disabled by specifying is_reparameterized = False as an argument of sample().

Returns:	A boolean indicating whether it is re-parameterized.
Return type:	bool

value_ndims¶

Get the number of value dimensions in samples.

Returns:	The number of value dimensions in samples.
Return type:	int

Methods Documentation

batch_ndims_to_value(ndims)¶

Convert the last few batch_ndims into value_ndims.

For a particular Distribution, the number of dimensions between the samples and the log-probability of the samples should satisfy:

samples.ndims - distribution.value_ndims == log_det.ndims

We denote samples.ndims - distribution.value_ndims by batch_ndims. This method thus wraps the current distribution, converts the last few batch_ndims into value_ndims.

Parameters:	ndims (int) – The last few batch_ndims to be converted into value_ndims. Must be non-negative.
Returns:	The converted distribution.
Return type:	Distribution

expand_value_ndims(ndims)¶

Convert the last few batch_ndims into value_ndims.

For a particular Distribution, the number of dimensions between the samples and the log-probability of the samples should satisfy:

samples.ndims - distribution.value_ndims == log_det.ndims

We denote samples.ndims - distribution.value_ndims by batch_ndims. This method thus wraps the current distribution, converts the last few batch_ndims into value_ndims.

Parameters:	ndims (int) – The last few batch_ndims to be converted into value_ndims. Must be non-negative.
Returns:	The converted distribution.
Return type:	Distribution

get_batch_shape()¶

Get the static batch shape of the samples.

Returns:	The batch shape.
Return type:	tf.TensorShape

log_prob(given, group_ndims=0, name=None)¶

Compute the log-densities of x against the distribution.

Parameters:	given (Tensor) – The samples to be tested. group_ndims (int or tf.Tensor) – If specified, the last group_ndims dimensions of the log-densities will be summed up. (default 0) name – TensorFlow name scope of the graph nodes. (default “log_prob”).
Returns:	The log-densities of given.
Return type:	tf.Tensor

prob(given, group_ndims=0, name=None)¶

Compute the densities of x against the distribution.

Parameters:	given (Tensor) – The samples to be tested. group_ndims (int or tf.Tensor) – If specified, the last group_ndims dimensions of the log-densities will be summed up. (default 0) name – TensorFlow name scope of the graph nodes. (default “prob”).
Returns:	The densities of given.
Return type:	tf.Tensor

sample(n_samples=None, group_ndims=0, is_reparameterized=None, compute_density=None, name=None)¶

Generate samples from the distribution.

Parameters:

n_samples (int or tf.Tensor or None) – A 0-D int32 Tensor or None. How many independent samples to draw from the distribution. The samples will have shape [n_samples] + batch_shape + value_shape, or batch_shape + value_shape if n_samples is None.
group_ndims (int or tf.Tensor) – Number of dimensions at the end of [n_samples] + batch_shape to be considered as events group. This will effect the behavior of log_prob() and prob(). (default 0)
is_reparameterized (bool) – If True, raises RuntimeError if the distribution is not re-parameterized. If False, disable re-parameterization even if the distribution is re-parameterized. (default None, following the setting of distribution)
compute_density (bool) – Whether or not to immediately compute the log-density for the samples? (default None, determine by the distribution class itself)
name – TensorFlow name scope of the graph nodes. (default “sample”).

Returns:

The samples as: StochasticTensor.

Return type:

tfsnippet.stochastic.StochasticTensor