MultiStepStateScheduler#

class ignite.handlers.state_param_scheduler.MultiStepStateScheduler(initial_value, gamma, milestones, param_name, save_history=False, create_new=False)[source]#

Update a parameter during training by using a multi step function. The function decays the parameter value by gamma once the number of steps reaches one of the milestones. Based on MultiStepLR from PyTorch. https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.MultiStepLR.html

Parameters

initial_value (float) – Starting value of the parameter.
gamma (float) – Multiplicative factor of parameter value decay.
milestones (List[int]) – List of step indices. Must be increasing.
param_name (str) – name of parameter to update.
save_history (bool) – whether to log the parameter values to engine.state.param_history, (default=False).
create_new (bool) – whether to create param_name on engine.state taking into account whether param_name attribute already exists or not. Overrides existing attribute by default, (default=False).

Examples

from collections import OrderedDict

import torch
from torch import nn, optim

from ignite.engine import *
from ignite.handlers import *
from ignite.metrics import *
from ignite.metrics.clustering import *
from ignite.metrics.regression import *
from ignite.utils import *

# create default evaluator for doctests

def eval_step(engine, batch):
    return batch

default_evaluator = Engine(eval_step)

# create default optimizer for doctests

param_tensor = torch.zeros([1], requires_grad=True)
default_optimizer = torch.optim.SGD([param_tensor], lr=0.1)

# create default trainer for doctests
# as handlers could be attached to the trainer,
# each test must define his own trainer using `.. testsetup:`

def get_default_trainer():

    def train_step(engine, batch):
        return batch

    return Engine(train_step)

# create default model for doctests

default_model = nn.Sequential(OrderedDict([
    ('base', nn.Linear(4, 2)),
    ('fc', nn.Linear(2, 1))
]))

manual_seed(666)

default_trainer = get_default_trainer()

param_scheduler = MultiStepStateScheduler(
    param_name="param", initial_value=1, gamma=0.9, milestones=[3, 6, 9, 12], create_new=True
)

# parameter is param, initial_value sets param to 1, gamma is set as 0.9
# Epoch 1 to 2, param does not change as milestone is 3
# Epoch 3, param changes from 1 to 1*0.9, param = 0.9
# Epoch 3 to 5, param does not change as milestone is 6
# Epoch 6, param changes from 0.9 to 0.9*0.9, param = 0.81
# Epoch 6 to 8, param does not change as milestone is 9
# Epoch 9, param changes from 0.81 to 0.81*0.9, param = 0.729
# Epoch 9 to 11, param does not change as milestone is 12
# Epoch 12, param changes from 0.729 to 0.729*0.9, param = 0.6561

param_scheduler.attach(default_trainer, Events.EPOCH_COMPLETED)

@default_trainer.on(Events.EPOCH_COMPLETED)
def print_param():
    print(default_trainer.state.param)

default_trainer.run([0], max_epochs=12)

New in version 0.4.7.

Methods

get_param

Method to get current parameter values

get_param()[source]#

Method to get current parameter values

Returns: list of params, or scalar param
Return type: Union[List[float], float]