tensorboard_logger#

TensorBoard logger and its helper handlers.

Classes

`GradsHistHandler`	Helper handler to log model's gradients as histograms.
`GradsScalarHandler`	Helper handler to log model's gradients as scalars.
`OptimizerParamsHandler`	Helper handler to log optimizer parameters
`OutputHandler`	Helper handler to log engine's output, engine's state attributes and/or metrics
`TensorboardLogger`	TensorBoard handler to log metrics, model/optimizer parameters, gradients during the training and validation.
`WeightsHistHandler`	Helper handler to log model's weights as histograms.
`WeightsScalarHandler`	Helper handler to log model's weights as scalars.

class ignite.contrib.handlers.tensorboard_logger.GradsHistHandler(model, tag=None)[source]#

Helper handler to log model’s gradients as histograms.

Parameters

model (Module) – model to log weights
tag (Optional[str]) – common title for all produced plots. For example, “generator”

Examples

from ignite.contrib.handlers.tensorboard_logger import *

# Create a logger
tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

# Attach the logger to the trainer to log model's weights norm after each iteration
tb_logger.attach(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    log_handler=GradsHistHandler(model)
)

class ignite.contrib.handlers.tensorboard_logger.GradsScalarHandler(model, reduction=<function norm>, tag=None)[source]#

Helper handler to log model’s gradients as scalars. Handler iterates over the gradients of named parameters of the model, applies reduction function to each parameter produce a scalar and then logs the scalar.

Parameters

model (Module) – model to log weights
reduction (Callable) – function to reduce parameters into scalar
tag (Optional[str]) – common title for all produced plots. For example, “generator”

Examples

from ignite.contrib.handlers.tensorboard_logger import *

# Create a logger
tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

# Attach the logger to the trainer to log model's weights norm after each iteration
tb_logger.attach(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    log_handler=GradsScalarHandler(model, reduction=torch.norm)
)

class ignite.contrib.handlers.tensorboard_logger.OptimizerParamsHandler(optimizer, param_name='lr', tag=None)[source]#

Helper handler to log optimizer parameters

Parameters

optimizer (Optimizer) – torch optimizer or any object with attribute param_groups as a sequence.
param_name (str) – parameter name
tag (Optional[str]) – common title for all produced plots. For example, “generator”

Examples

from ignite.contrib.handlers.tensorboard_logger import *

# Create a logger
tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

# Attach the logger to the trainer to log optimizer's parameters, e.g. learning rate at each iteration
tb_logger.attach(
    trainer,
    log_handler=OptimizerParamsHandler(optimizer),
    event_name=Events.ITERATION_STARTED
)
# or equivalently
tb_logger.attach_opt_params_handler(
    trainer,
    event_name=Events.ITERATION_STARTED,
    optimizer=optimizer
)

class ignite.contrib.handlers.tensorboard_logger.OutputHandler(tag, metric_names=None, output_transform=None, global_step_transform=None, state_attributes=None)[source]#

Helper handler to log engine’s output, engine’s state attributes and/or metrics

Parameters

tag (str) – common title for all produced plots. For example, “training”
metric_names (Optional[List[str]]) – list of metric names to plot or a string “all” to plot all available metrics.
output_transform (Optional[Callable]) – output transform function to prepare engine.state.output as a number. For example, output_transform = lambda output: output This function can also return a dictionary, e.g {“loss”: loss1, “another_loss”: loss2} to label the plot with corresponding keys.
global_step_transform (Optional[Callable]) – global step transform function to output a desired global step. Input of the function is (engine, event_name). Output of function should be an integer. Default is None, global_step based on attached engine. If provided, uses function output as global_step. To setup global step from another engine, please use global_step_from_engine().
state_attributes (Optional[List[str]]) – list of attributes of the trainer.state to plot.

Examples

from ignite.contrib.handlers.tensorboard_logger import *

# Create a logger
tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

# Attach the logger to the evaluator on the validation dataset and log NLL, Accuracy metrics after
# each epoch. We setup `global_step_transform=global_step_from_engine(trainer)` to take the epoch
# of the `trainer`:
tb_logger.attach(
    evaluator,
    log_handler=OutputHandler(
        tag="validation",
        metric_names=["nll", "accuracy"],
        global_step_transform=global_step_from_engine(trainer)
    ),
    event_name=Events.EPOCH_COMPLETED
)
# or equivalently
tb_logger.attach_output_handler(
    evaluator,
    event_name=Events.EPOCH_COMPLETED,
    tag="validation",
    metric_names=["nll", "accuracy"],
    global_step_transform=global_step_from_engine(trainer)
)

Another example, where model is evaluated every 500 iterations:

from ignite.contrib.handlers.tensorboard_logger import *

@trainer.on(Events.ITERATION_COMPLETED(every=500))
def evaluate(engine):
    evaluator.run(validation_set, max_epochs=1)

tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

def global_step_transform(*args, **kwargs):
    return trainer.state.iteration

# Attach the logger to the evaluator on the validation dataset and log NLL, Accuracy metrics after
# every 500 iterations. Since evaluator engine does not have access to the training iteration, we
# provide a global_step_transform to return the trainer.state.iteration for the global_step, each time
# evaluator metrics are plotted on Tensorboard.

tb_logger.attach_output_handler(
    evaluator,
    event_name=Events.EPOCH_COMPLETED,
    tag="validation",
    metrics=["nll", "accuracy"],
    global_step_transform=global_step_transform
)

Another example where the State Attributes trainer.state.alpha and trainer.state.beta are also logged along with the NLL and Accuracy after each iteration:

tb_logger.attach(
    trainer,
    log_handler=OutputHandler(
        tag="training",
        metric_names=["nll", "accuracy"],
        state_attributes=["alpha", "beta"],
    ),
    event_name=Events.ITERATION_COMPLETED
)

Example of global_step_transform:

def global_step_transform(engine, event_name):
    return engine.state.get_event_attrib_value(event_name)

Changed in version 0.4.7: accepts an optional list of state_attributes

class ignite.contrib.handlers.tensorboard_logger.TensorboardLogger(*args, **kwargs)[source]#

TensorBoard handler to log metrics, model/optimizer parameters, gradients during the training and validation.

By default, this class favors tensorboardX package if installed:

pip install tensorboardX

otherwise, it falls back to using PyTorch’s SummaryWriter (>=v1.2.0).

Parameters

args (Any) – Positional arguments accepted from SummaryWriter.
kwargs (Any) –
Keyword arguments accepted from SummaryWriter. For example, log_dir to setup path to the directory where to log.

Examples

from ignite.contrib.handlers.tensorboard_logger import *

# Create a logger
tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

# Attach the logger to the trainer to log training loss at each iteration
tb_logger.attach_output_handler(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    tag="training",
    output_transform=lambda loss: {"loss": loss}
)

# Attach the logger to the evaluator on the training dataset and log NLL, Accuracy metrics after each epoch
# We setup `global_step_transform=global_step_from_engine(trainer)` to take the epoch
# of the `trainer` instead of `train_evaluator`.
tb_logger.attach_output_handler(
    train_evaluator,
    event_name=Events.EPOCH_COMPLETED,
    tag="training",
    metric_names=["nll", "accuracy"],
    global_step_transform=global_step_from_engine(trainer),
)

# Attach the logger to the evaluator on the validation dataset and log NLL, Accuracy metrics after
# each epoch. We setup `global_step_transform=global_step_from_engine(trainer)` to take the epoch of the
# `trainer` instead of `evaluator`.
tb_logger.attach_output_handler(
    evaluator,
    event_name=Events.EPOCH_COMPLETED,
    tag="validation",
    metric_names=["nll", "accuracy"],
    global_step_transform=global_step_from_engine(trainer)),
)

# Attach the logger to the trainer to log optimizer's parameters, e.g. learning rate at each iteration
tb_logger.attach_opt_params_handler(
    trainer,
    event_name=Events.ITERATION_STARTED,
    optimizer=optimizer,
    param_name='lr'  # optional
)

# Attach the logger to the trainer to log model's weights norm after each iteration
tb_logger.attach(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    log_handler=WeightsScalarHandler(model)
)

# Attach the logger to the trainer to log model's weights as a histogram after each epoch
tb_logger.attach(
    trainer,
    event_name=Events.EPOCH_COMPLETED,
    log_handler=WeightsHistHandler(model)
)

# Attach the logger to the trainer to log model's gradients norm after each iteration
tb_logger.attach(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    log_handler=GradsScalarHandler(model)
)

# Attach the logger to the trainer to log model's gradients as a histogram after each epoch
tb_logger.attach(
    trainer,
    event_name=Events.EPOCH_COMPLETED,
    log_handler=GradsHistHandler(model)
)

# We need to close the logger when we are done
tb_logger.close()

It is also possible to use the logger as context manager:

from ignite.contrib.handlers.tensorboard_logger import *

with TensorboardLogger(log_dir="experiments/tb_logs") as tb_logger:

    trainer = Engine(update_fn)
    # Attach the logger to the trainer to log training loss at each iteration
    tb_logger.attach_output_handler(
        trainer,
        event_name=Events.ITERATION_COMPLETED,
        tag="training",
        output_transform=lambda loss: {"loss": loss}
    )

class ignite.contrib.handlers.tensorboard_logger.WeightsHistHandler(model, tag=None)[source]#

Helper handler to log model’s weights as histograms.

Parameters

model (Module) – model to log weights
tag (Optional[str]) – common title for all produced plots. For example, “generator”

Examples

from ignite.contrib.handlers.tensorboard_logger import *

# Create a logger
tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

# Attach the logger to the trainer to log model's weights norm after each iteration
tb_logger.attach(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    log_handler=WeightsHistHandler(model)
)

class ignite.contrib.handlers.tensorboard_logger.WeightsScalarHandler(model, reduction=<function norm>, tag=None)[source]#

Helper handler to log model’s weights as scalars. Handler iterates over named parameters of the model, applies reduction function to each parameter produce a scalar and then logs the scalar.

Parameters

model (Module) – model to log weights
reduction (Callable) – function to reduce parameters into scalar
tag (Optional[str]) – common title for all produced plots. For example, “generator”

Examples

from ignite.contrib.handlers.tensorboard_logger import *

# Create a logger
tb_logger = TensorboardLogger(log_dir="experiments/tb_logs")

# Attach the logger to the trainer to log model's weights norm after each iteration
tb_logger.attach(
    trainer,
    event_name=Events.ITERATION_COMPLETED,
    log_handler=WeightsScalarHandler(model, reduction=torch.norm)
)

ignite.contrib.handlers.tensorboard_logger.global_step_from_engine(engine, custom_event_name=None)[source]#

Helper method to setup global_step_transform function using another engine. This can be helpful for logging trainer epoch/iteration while output handler is attached to an evaluator.

Parameters

engine (Engine) – engine which state is used to provide the global step
custom_event_name (Optional[Events]) – registered event name. Optional argument, event name to use.

Returns

global step based on provided engine

Return type

Callable