tfsnippet.distributions¶

class tfsnippet.distributions.Distribution¶

Bases: object

Base class for probability distributions.

A Distribution object receives inputs as distribution parameters, generating samples and computing densities according to these inputs. The shape of the inputs can have more dimensions than the nature shape of the distribution parameters, since Distribution is designed to work with batch parameters, samples and densities.

The shape of the parameters of a Distribution object would be decomposed into batch_shape + param_shape, with param_shape being the nature shape of the parameter. For example, a 5-class Categorical distribution with class probabilities of shape (3, 4, 5) would have (3, 4) as the batch_shape, with (5,) as the param_shape, corresponding to the probabilities of 5 classes.

Generating n samples from a Distribution object would result in tensors with shape [n] (sample_shape) + batch_shape + value_shape, with value_shape being the nature shape of an individual sample from the distribution. For example, the value_shape of a Categorical is (), such that the sample shape would be (3, 4), provided the shape of class probabilities is (3, 4, 5).

Computing the densities (i.e., prob(x) or log_prob(x)) of samples involves broadcasting these samples against the distribution parameters. These samples should be broadcastable against batch_shape + value_shape. Suppose the shape of the samples can be decomposed into sample_shape + batch_shape + value_shape, then by default, the shape of the densities should be sample_shape + batch_shape, i.e., each individual sample resulting in an individual density value.

batch_shape¶

Get the batch shape of the samples.

Returns:	The batch shape as tensor.
Return type:	tf.Tensor

dtype¶

Get the data type of samples.

Returns:	Data type of the samples.
Return type:	tf.DType

classmethod factory(**kwargs)¶

Get a factory for constructing a distribution instance of cls, with some default parameters.

factory = Normal.factory(std=1.)

# equivalent to Normal(mean=0., std=1.)
normal = factory(mean=0.)

# override `std`
normal = factory(mean=1., std=2.)

# parameters can also be specified via a dict
normal = factory({'mean': 0.})

Parameters:	**kwargs – The default named arguments.
Returns:	The distribution factory instance.
Return type:	tfsnippet.distributions.DistributionFactory

get_batch_shape()¶

Get the static batch shape of the samples.

Returns:	The batch shape.
Return type:	tf.TensorShape

get_value_shape()¶

Get the static value shape of an individual sample.

Returns:	The static value shape.
Return type:	tf.TensorShape

is_continuous¶

Whether or not the distribution is continuous?

Returns:	A boolean indicating whether it is continuous.
Return type:	bool

is_reparameterized¶

Whether or not the distribution is re-parameterized?

The re-parameterization trick is proposed in “Auto-Encoding Variational Bayes” (Kingma, D.P. and Welling), allowing the gradients to be propagated back along the samples. Note that the re-parameterization can be disabled by specifying is_reparameterized = False as an argument of sample().

Returns:	A boolean indicating whether it is re-parameterized.
Return type:	bool

log_prob(given, group_ndims=0, name=None)¶

Compute the log-densities of x against the distribution.

Parameters:	given (Tensor) – The samples to be tested. group_ndims (int or tf.Tensor) – If specified, the last group_ndims dimensions of the log-densities will be summed up. (default 0) name (str) – TensorFlow name scope of the graph nodes. (default “log_prob”).
Returns:	The log-densities of given.
Return type:	tf.Tensor

prob(given, group_ndims=0, name=None)¶

Compute the densities of x against the distribution.

Parameters:	given (Tensor) – The samples to be tested. group_ndims (int or tf.Tensor) – If specified, the last group_ndims dimensions of the log-densities will be summed up. (default 0) name (str) – TensorFlow name scope of the graph nodes. (default “prob”).
Returns:	The densities of given.
Return type:	tf.Tensor

sample(n_samples=None, group_ndims=0, is_reparameterized=None, compute_density=None, name=None)¶

Generate samples from the distribution.

Parameters:

n_samples (int or tf.Tensor or None) – A 0-D int32 Tensor or None. How many independent samples to draw from the distribution. The samples will have shape [n_samples] + batch_shape + value_shape, or batch_shape + value_shape if n_samples is None.
group_ndims (int or tf.Tensor) – Number of dimensions at the end of [n_samples] + batch_shape to be considered as events group. This will effect the behavior of log_prob() and prob(). (default 0)
is_reparameterized (bool) – If True, raises RuntimeError if the distribution is not re-parameterized. If False, disable re-parameterization even if the distribution is re-parameterized. (default None, following the setting of distribution)
compute_density (bool) – Whether or not to immediately compute the log-density for the samples? (default None, determine by the distribution class itself)
name (str) – TensorFlow name scope of the graph nodes. (default “sample”).

Returns:

The samples as: StochasticTensor.

Return type:

tfsnippet.stochastic.StochasticTensor

value_shape¶

Get the value shape of an individual sample.

Returns:	The value shape as tensor.
Return type:	tf.Tensor

class tfsnippet.distributions.DistributionFactory(distribution_class, default_args=None)¶

Bases: object

A factory for constructing a Distribution instance.

__call__(distribution_params=None, **kwargs)¶

Construct a Distribution.

Parameters:	distribution_params (dict[str, any]) – Dict of distribution parameters for constructing the distribution. Usually used for consuming the output of a `tfsnippet.modules.DictMapper`. Will override the default arguments in default_args. **kwargs – Other named arguments for constructing the distribution. Will override the arguments in default_args and in distribution_params.
Returns:	The constructed distribution instance.
Return type:	Distribution

__init__(distribution_class, default_args=None)¶

Construct the DistributionFactory.

Parameters:	distribution_class – The class of the distribution to be constructed, a subclass of `Distribution`. default_args (dict[str, any]) – Dict of default named arguments for constructing the distribution.

class tfsnippet.distributions.OnehotCategorical(logits, dtype=None)¶

Bases: tfsnippet.distributions.wrapper.ZhuSuanDistribution

One-hot multivariate Categorical distribution.

A batch of samples is an N-D Tensor with dtype values in range [0, n_categories).

See also

tfsnippet.distributions.Distribution, zhusuan.distributions.Distribution, zhusuan.distributions.OnehotCategorical

__init__(logits, dtype=None)¶

Construct the OnehotCategorical.

Parameters:	logits – An N-D (N >= 1) float Tensor of shape `(..., n_categories)`. Each slice [i, j,…, k, :] represents the un-normalized log-probabilities for all categories. \(\mathrm{logits} \propto \log p\) dtype – The value type of samples from the distribution. (default `tf.int32`)

logits¶: The un-normalized log probabilities.

n_categories¶: The number of categories in the distribution.

class tfsnippet.distributions.Concrete(temperature, logits, is_reparameterized=True, check_numerics=False)¶

Bases: tfsnippet.distributions.wrapper.ZhuSuanDistribution

The class of Concrete (or Gumbel-Softmax) distribution from (Maddison, 2016; Jang, 2016), served as the continuous relaxation of the OnehotCategorical.

See also

tfsnippet.distributions.Distribution, zhusuan.distributions.Distribution, zhusuan.distributions.Concrete

__init__(temperature, logits, is_reparameterized=True, check_numerics=False)¶

Construct the ExpConcrete.

Parameters:	temperature – A 0-D float Tensor. The temperature of the relaxed distribution. The temperature should be positive. logits – An N-D (N >= 1) float Tensor of shape `(..., n_categories)`. Each slice [i, j,…, k, :] represents the un-normalized log probabilities for all categories. \(\mathrm{logits} \propto \log p\)

logits¶: The un-normalized log probabilities.

n_categories¶: The number of categories in the distribution.

temperature¶: The temperature of this concrete distribution.

class tfsnippet.distributions.ExpConcrete(temperature, logits, is_reparameterized=True, check_numerics=False)¶

Bases: tfsnippet.distributions.wrapper.ZhuSuanDistribution

The class of ExpConcrete distribution from (Maddison, 2016), transformed from Concrete by taking logarithm.

See also

tfsnippet.distributions.Distribution, zhusuan.distributions.Distribution, zhusuan.distributions.ExpConcrete

__init__(temperature, logits, is_reparameterized=True, check_numerics=False)¶

Construct the ExpConcrete.

Parameters:	temperature – A 0-D float Tensor. The temperature of the relaxed distribution. The temperature should be positive. logits – An N-D (N >= 1) float Tensor of shape `(..., n_categories)`. Each slice [i, j,…, k, :] represents the un-normalized log probabilities for all categories. \(\mathrm{logits} \propto \log p\)

logits¶: The un-normalized log probabilities.

n_categories¶: The number of categories in the distribution.

temperature¶: The temperature of this concrete distribution.

class tfsnippet.distributions.Normal(mean, std=None, logstd=None, is_reparameterized=True, check_numerics=False)¶

Bases: tfsnippet.distributions.wrapper.ZhuSuanDistribution

Univariate Normal distribution.

See also

tfsnippet.distributions.Distribution, zhusuan.distributions.Distribution, zhusuan.distributions.Normal

__init__(mean, std=None, logstd=None, is_reparameterized=True, check_numerics=False)¶

Construct the Normal.

Parameters:

mean – A float tensor, the mean of the Normal distribution. Should be broadcastable against std / logstd.
std – A float tensor, the standard deviation of the Normal distribution. Should be positive, and broadcastable against mean. One and only one of std or logstd should be specified.
logstd – A float tensor, the log standard deviation of the Normal distribution. Should be broadcastable against mean.
is_reparameterized (bool) – Whether or not the gradients can be propagated through parameters? (default True)
check_numerics (bool) – Whether or not to check numeric issues.

logstd¶: Get the log standard deviation of the Normal distribution.

mean¶: Get the mean of the Normal distribution.

std¶: Get the standard deviation of the Normal distribution.

class tfsnippet.distributions.Bernoulli(logits, dtype=tf.int32)¶

Bases: tfsnippet.distributions.wrapper.ZhuSuanDistribution

Univariate Bernoulli distribution.

See also

tfsnippet.distributions.Distribution, zhusuan.distributions.Distribution, zhusuan.distributions.Bernoulli

__init__(logits, dtype=tf.int32)¶

Construct the Bernoulli.

Parameters:	logits – A float tensor, log-odds of probabilities of being 1. \(\mathrm{logits} = \log \frac{p}{1 - p}\) dtype – The value type of samples from the distribution. (default `tf.int32`)

logits¶: The log-odds of probabilities of being 1.

class tfsnippet.distributions.Categorical(logits, dtype=None)¶

Bases: tfsnippet.distributions.wrapper.ZhuSuanDistribution

Univariate Categorical distribution.

A batch of samples is an (N-1)-D Tensor with dtype values in range [0, n_categories).

See also

tfsnippet.distributions.Distribution, zhusuan.distributions.Distribution, zhusuan.distributions.Categorical

__init__(logits, dtype=None)¶

Construct the Categorical.

Parameters:	logits – An N-D (N >= 1) float Tensor of shape `(..., n_categories)`. Each slice [i, j,…, k, :] represents the un-normalized log probabilities for all categories. \(\mathrm{logits} \propto \log p\) dtype – The value type of samples from the distribution. (default `tf.int32`)

logits¶: The un-normalized log probabilities.

n_categories¶: The number of categories in the distribution.

tfsnippet.distributions.Discrete¶: alias of tfsnippet.distributions.univariate.Categorical

class tfsnippet.distributions.Uniform(minval=0.0, maxval=1.0, is_reparameterized=True, check_numerics=False)¶

Bases: tfsnippet.distributions.wrapper.ZhuSuanDistribution

Univariate Uniform distribution.

See also

tfsnippet.distributions.Distribution, zhusuan.distributions.Distribution, zhusuan.distributions.Uniform

__init__(minval=0.0, maxval=1.0, is_reparameterized=True, check_numerics=False)¶

Construct the Uniform.

Parameters:

minval – A float Tensor. The lower bound on the range of the uniform distribution. Should be broadcastable to match maxval.
maxval – A float Tensor. The upper bound on the range of the uniform distribution. Should be element-wise bigger than minval.
is_reparameterized (bool) – Whether or not the gradients can be propagated through parameters? (default True)
check_numerics (bool) – Whether or not to check numeric issues.

maxval¶: The upper bound on the range of the uniform distribution.

minval¶: The lower bound on the range of the uniform distribution.

tfsnippet.distributions.validate_group_ndims(group_ndims, name=None)¶

Validate the specified value for group_ndims argument.

If the specified group_ndims is a dynamic Tensor, additional assertion will be added to the graph node of group_ndims.

Parameters:	group_ndims – Object to be validated. name – TensorFlow name scope of the graph nodes. (default “validate_group_ndims”)
Returns:	The validated group_ndims.
Return type:	tf.Tensor or int
Raises:	`ValueError` – If the specified group_ndims cannot pass validation.

tfsnippet.distributions.reduce_group_ndims(operation, tensor, group_ndims, name=None)¶

Reduce the last group_ndims dimensions in tensor, using operation.

In Distribution, when computing the (log-)densities of certain tensor, the last few dimensions may represent a group of events, thus should be accounted together. This method can be used to reduce these dimensions, for example:

log_prob = reduce_group_ndims(tf.reduce_sum, log_prob, group_ndims)
prob = reduce_group_ndims(tf.reduce_prod, log_prob, group_ndims)

Parameters:	operation – The operation for reducing the last group_ndims dimensions. It must receive tensor as the 1st argument, and axis as the 2nd argument. tensor – The tensor to be reduced. group_ndims – The number of dimensions at the end of tensor to be reduced. If it is a constant integer and is zero, then no operation will take place. name – TensorFlow name scope of the graph nodes. (default “reduce_group_ndims”)
Returns:	The reduced tensor.
Return type:	tf.Tensor
Raises:	`ValueError` – If group_ndims cannot be validated by `validate_group_ndims()`.

tfsnippet.distributions.as_distribution(distribution)¶

Convert a supported type of distribution into Distribution type.

Parameters:	distribution – A supported distribution instance. Supported types are: 1. `Distribution`, 2. `zhusuan.distributions.Distribution`.
Returns:	The wrapped distribution of `Distribution` type.
Return type:	Distribution
Raises:	`TypeError` – If the specified distribution cannot be converted.

tfsnippet.distributions¶

TFSnippet

Navigation

Related Topics