Evaluator¶

class tfsnippet.Evaluator(loop, metrics, inputs, data_flow, feed_dict=None, time_metric_name='eval_time', batch_weight_func=<function auto_batch_weight>)¶

Bases: object

Class to compute evaluation metrics.

It is a common practice to compute one or more metrics for evaluation and validation during the training process. This class provides a convenient interface for computing metrics by mini-batches.

The event schedule of an Evaluator can be briefly described as follows:

events.fire(EventKeys.BEFORE_EXECUTION, self)

...  # actually run the evaluation

events.reverse_fire(EventKeys.AFTER_EXECUTION, self)

Attributes Summary

`batch_weight_func`	Get the function to compute the metric weight for each mini-batch.
`data_flow`	Get the validation data flow.
`events`	Get the event source object.
`feed_dict`	Get the fixed feed dict.
`inputs`	Get the input placeholders.
`last_metrics_dict`	Get the metric values from last evaluation.
`loop`	Get the training loop object.
`metrics`	Get the metrics to compute.
`time_metric_name`	Get the metric name for collecting evaluation time usage.

Methods Summary

run([feed_dict]) Run evaluation.

Attributes Documentation

batch_weight_func¶: Get the function to compute the metric weight for each mini-batch.

data_flow¶

Get the validation data flow.

Returns:	The validation data flow.
Return type:	DataFlow

events¶

Get the event source object.

Returns:	The event source object.
Return type:	EventSource

feed_dict¶

Get the fixed feed dict.

Returns:	The fixed feed dict.
Return type:	dict[tf.Tensor, any]

inputs¶

Get the input placeholders.

Returns:	The input placeholders.
Return type:	list[tf.Tensor]

last_metrics_dict¶

Get the metric values from last evaluation.

Returns:	The metric values dict.
Return type:	dict[str, any]

loop¶

Get the training loop object.

Returns:	The training loop object.
Return type:	TrainLoop

metrics¶

Get the metrics to compute.

Returns:	The metrics to compute.
Return type:	OrderedDict[str, tf.Tensor]

time_metric_name¶: Get the metric name for collecting evaluation time usage.

Methods Documentation

run(feed_dict=None)¶

Run evaluation.

Parameters:	feed_dict – The extra feed dict to be merged with the already configured dict. (default `None`)