tensorboard_logger#
TensorBoard logger and its helper handlers.
Classes
Helper handler to log model's gradients as histograms. |
|
Helper handler to log model's gradients as scalars. |
|
Helper handler to log optimizer parameters |
|
Helper handler to log engine's output, engine's state attributes and/or metrics |
|
TensorBoard handler to log metrics, model/optimizer parameters, gradients during the training and validation. |
|
Helper handler to log model's weights as histograms. |
|
Helper handler to log model's weights as scalars. |
- class ignite.handlers.tensorboard_logger.GradsHistHandler(model, tag=None, whitelist=None)[source]#
Helper handler to log model’s gradients as histograms.
- Parameters
model (Module) – model to log weights
tag (Optional[str]) – common title for all produced plots. For example, “generator”
whitelist (Optional[Union[List[str], Callable[[str, Parameter], bool]]]) – specific gradients to log. Should be list of model’s submodules or parameters names, or a callable which gets weight along with its name and determines if its gradient should be logged. Names should be fully-qualified. For more information please refer to PyTorch docs. If not given, all of model’s gradients are logged.
Examples
from ignite.handlers.tensorboard_logger import * # Create a logger tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Attach the logger to the trainer to log model's weights norm after each iteration tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=GradsHistHandler(model) )
from ignite.handlers.tensorboard_logger import * tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Log gradient of `fc.bias` tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=GradsHistHandler(model, whitelist=['fc.bias']) )
from ignite.handlers.tensorboard_logger import * tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Log gradient of weights which have shape (2, 1) def has_shape_2_1(n, p): return p.shape == (2,1) tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=GradsHistHandler(model, whitelist=has_shape_2_1) )
Changed in version 0.4.9: optional argument whitelist added.
- class ignite.handlers.tensorboard_logger.GradsScalarHandler(model, reduction=<function norm>, tag=None, whitelist=None)[source]#
Helper handler to log model’s gradients as scalars. Handler, upon construction, iterates over named parameters of the model and keep reference to ones permitted by the whitelist. Then at every call, applies reduction function to each parameter’s gradient, produces a scalar and logs it.
- Parameters
model (Module) – model to log weights
reduction (Callable[[Tensor], Union[float, Tensor]]) – function to reduce parameters into scalar
tag (Optional[str]) – common title for all produced plots. For example, “generator”
whitelist (Optional[Union[List[str], Callable[[str, Parameter], bool]]]) –
specific gradients to log. Should be list of model’s submodules or parameters names, or a callable which gets weight along with its name and determines if its gradient should be logged. Names should be fully-qualified. For more information please refer to PyTorch docs. If not given, all of model’s gradients are logged.
Examples
from ignite.handlers.tensorboard_logger import * # Create a logger tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Attach the logger to the trainer to log model's gradients norm after each iteration tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=GradsScalarHandler(model, reduction=torch.norm) )
from ignite.handlers.tensorboard_logger import * tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Log gradient of `base` tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=GradsScalarHandler( model, reduction=torch.norm, whitelist=['base'] ) )
from ignite.handlers.tensorboard_logger import * tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Log gradient of weights which belong to a `fc` layer def is_in_fc_layer(n, p): return 'fc' in n tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=GradsScalarHandler(model, whitelist=is_in_fc_layer) )
Changed in version 0.4.9: optional argument whitelist added.
- class ignite.handlers.tensorboard_logger.OptimizerParamsHandler(optimizer, param_name='lr', tag=None)[source]#
Helper handler to log optimizer parameters
- Parameters
Examples
from ignite.handlers.tensorboard_logger import * # Create a logger tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Attach the logger to the trainer to log optimizer's parameters, e.g. learning rate at each iteration tb_logger.attach( trainer, log_handler=OptimizerParamsHandler(optimizer), event_name=Events.ITERATION_STARTED ) # or equivalently tb_logger.attach_opt_params_handler( trainer, event_name=Events.ITERATION_STARTED, optimizer=optimizer )
- class ignite.handlers.tensorboard_logger.OutputHandler(tag, metric_names=None, output_transform=None, global_step_transform=None, state_attributes=None)[source]#
Helper handler to log engine’s output, engine’s state attributes and/or metrics
- Parameters
tag (str) – common title for all produced plots. For example, “training”
metric_names (Optional[List[str]]) – list of metric names to plot or a string “all” to plot all available metrics.
output_transform (Optional[Callable]) – output transform function to prepare engine.state.output as a number. For example, output_transform = lambda output: output This function can also return a dictionary, e.g {“loss”: loss1, “another_loss”: loss2} to label the plot with corresponding keys.
global_step_transform (Optional[Callable[[Engine, Union[str, Events]], int]]) – global step transform function to output a desired global step. Input of the function is (engine, event_name). Output of function should be an integer. Default is None, global_step based on attached engine. If provided, uses function output as global_step. To setup global step from another engine, please use
global_step_from_engine()
.state_attributes (Optional[List[str]]) – list of attributes of the
trainer.state
to plot.
Examples
from ignite.handlers.tensorboard_logger import * # Create a logger tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Attach the logger to the evaluator on the validation dataset and log NLL, Accuracy metrics after # each epoch. We setup `global_step_transform=global_step_from_engine(trainer)` to take the epoch # of the `trainer`: tb_logger.attach( evaluator, log_handler=OutputHandler( tag="validation", metric_names=["nll", "accuracy"], global_step_transform=global_step_from_engine(trainer) ), event_name=Events.EPOCH_COMPLETED ) # or equivalently tb_logger.attach_output_handler( evaluator, event_name=Events.EPOCH_COMPLETED, tag="validation", metric_names=["nll", "accuracy"], global_step_transform=global_step_from_engine(trainer) )
Another example, where model is evaluated every 500 iterations:
from ignite.handlers.tensorboard_logger import * @trainer.on(Events.ITERATION_COMPLETED(every=500)) def evaluate(engine): evaluator.run(validation_set, max_epochs=1) tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") def global_step_transform(*args, **kwargs): return trainer.state.iteration # Attach the logger to the evaluator on the validation dataset and log NLL, Accuracy metrics after # every 500 iterations. Since evaluator engine does not have access to the training iteration, we # provide a global_step_transform to return the trainer.state.iteration for the global_step, each time # evaluator metrics are plotted on Tensorboard. tb_logger.attach_output_handler( evaluator, event_name=Events.EPOCH_COMPLETED, tag="validation", metrics=["nll", "accuracy"], global_step_transform=global_step_transform )
Another example where the State Attributes
trainer.state.alpha
andtrainer.state.beta
are also logged along with the NLL and Accuracy after each iteration:tb_logger.attach( trainer, log_handler=OutputHandler( tag="training", metric_names=["nll", "accuracy"], state_attributes=["alpha", "beta"], ), event_name=Events.ITERATION_COMPLETED )
Example of global_step_transform:
def global_step_transform(engine, event_name): return engine.state.get_event_attrib_value(event_name)
Changed in version 0.4.7: accepts an optional list of state_attributes
- class ignite.handlers.tensorboard_logger.TensorboardLogger(*args, **kwargs)[source]#
TensorBoard handler to log metrics, model/optimizer parameters, gradients during the training and validation.
By default, this class favors tensorboardX package if installed:
pip install tensorboardX
otherwise, it falls back to using PyTorch’s SummaryWriter (>=v1.2.0).
- Parameters
args (Any) – Positional arguments accepted from SummaryWriter.
kwargs (Any) –
Keyword arguments accepted from SummaryWriter. For example, log_dir to setup path to the directory where to log.
Examples
from ignite.handlers.tensorboard_logger import * # Create a logger tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Attach the logger to the trainer to log training loss at each iteration tb_logger.attach_output_handler( trainer, event_name=Events.ITERATION_COMPLETED, tag="training", output_transform=lambda loss: {"loss": loss} ) # Attach the logger to the evaluator on the training dataset and log NLL, Accuracy metrics after each epoch # We setup `global_step_transform=global_step_from_engine(trainer)` to take the epoch # of the `trainer` instead of `train_evaluator`. tb_logger.attach_output_handler( train_evaluator, event_name=Events.EPOCH_COMPLETED, tag="training", metric_names=["nll", "accuracy"], global_step_transform=global_step_from_engine(trainer), ) # Attach the logger to the evaluator on the validation dataset and log NLL, Accuracy metrics after # each epoch. We setup `global_step_transform=global_step_from_engine(trainer)` to take the epoch of the # `trainer` instead of `evaluator`. tb_logger.attach_output_handler( evaluator, event_name=Events.EPOCH_COMPLETED, tag="validation", metric_names=["nll", "accuracy"], global_step_transform=global_step_from_engine(trainer)), ) # Attach the logger to the trainer to log optimizer's parameters, e.g. learning rate at each iteration tb_logger.attach_opt_params_handler( trainer, event_name=Events.ITERATION_STARTED, optimizer=optimizer, param_name='lr' # optional ) # Attach the logger to the trainer to log model's weights norm after each iteration tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=WeightsScalarHandler(model) ) # Attach the logger to the trainer to log model's weights as a histogram after each epoch tb_logger.attach( trainer, event_name=Events.EPOCH_COMPLETED, log_handler=WeightsHistHandler(model) ) # Attach the logger to the trainer to log model's gradients norm after each iteration tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=GradsScalarHandler(model) ) # Attach the logger to the trainer to log model's gradients as a histogram after each epoch tb_logger.attach( trainer, event_name=Events.EPOCH_COMPLETED, log_handler=GradsHistHandler(model) ) # We need to close the logger when we are done tb_logger.close()
It is also possible to use the logger as context manager:
from ignite.handlers.tensorboard_logger import * with TensorboardLogger(log_dir="experiments/tb_logs") as tb_logger: trainer = Engine(update_fn) # Attach the logger to the trainer to log training loss at each iteration tb_logger.attach_output_handler( trainer, event_name=Events.ITERATION_COMPLETED, tag="training", output_transform=lambda loss: {"loss": loss} )
- class ignite.handlers.tensorboard_logger.WeightsHistHandler(model, tag=None, whitelist=None)[source]#
Helper handler to log model’s weights as histograms.
- Parameters
model (Module) – model to log weights
tag (Optional[str]) – common title for all produced plots. For example, “generator”
whitelist (Optional[Union[List[str], Callable[[str, Parameter], bool]]]) –
specific weights to log. Should be list of model’s submodules or parameters names, or a callable which gets weight along with its name and determines if it should be logged. Names should be fully-qualified. For more information please refer to PyTorch docs. If not given, all of model’s weights are logged.
Examples
from ignite.handlers.tensorboard_logger import * # Create a logger tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Attach the logger to the trainer to log model's weights norm after each iteration tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=WeightsHistHandler(model) )
from ignite.handlers.tensorboard_logger import * tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Log weights of `fc` layer weights = ['fc'] # Attach the logger to the trainer to log weights norm after each iteration tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=WeightsHistHandler(model, whitelist=weights) )
from ignite.handlers.tensorboard_logger import * tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Log weights which name include 'conv'. weight_selector = lambda name, p: 'conv' in name # Attach the logger to the trainer to log weights norm after each iteration tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=WeightsHistHandler(model, whitelist=weight_selector) )
Changed in version 0.4.9: optional argument whitelist added.
- class ignite.handlers.tensorboard_logger.WeightsScalarHandler(model, reduction=<function norm>, tag=None, whitelist=None)[source]#
Helper handler to log model’s weights as scalars. Handler, upon construction, iterates over named parameters of the model and keep reference to ones permitted by whitelist. Then at every call, applies reduction function to each parameter, produces a scalar and logs it.
- Parameters
model (Module) – model to log weights
reduction (Callable[[Tensor], Union[float, Tensor]]) – function to reduce parameters into scalar
tag (Optional[str]) – common title for all produced plots. For example, “generator”
whitelist (Optional[Union[List[str], Callable[[str, Parameter], bool]]]) –
specific weights to log. Should be list of model’s submodules or parameters names, or a callable which gets weight along with its name and determines if it should be logged. Names should be fully-qualified. For more information please refer to PyTorch docs. If not given, all of model’s weights are logged.
Examples
from ignite.handlers.tensorboard_logger import * # Create a logger tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Attach the logger to the trainer to log model's weights norm after each iteration tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=WeightsScalarHandler(model, reduction=torch.norm) )
from ignite.handlers.tensorboard_logger import * tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Log only `fc` weights tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=WeightsScalarHandler( model, whitelist=['fc'] ) )
from ignite.handlers.tensorboard_logger import * tb_logger = TensorboardLogger(log_dir="experiments/tb_logs") # Log weights which have `bias` in their names def has_bias_in_name(n, p): return 'bias' in n tb_logger.attach( trainer, event_name=Events.ITERATION_COMPLETED, log_handler=WeightsScalarHandler(model, whitelist=has_bias_in_name) )
Changed in version 0.4.9: optional argument whitelist added.