ParamGroupScheduler

class ignite.handlers.param_scheduler.ParamGroupScheduler(schedulers, names=None, save_history=False)

Scheduler helper to group multiple schedulers into one.

Parameters

• schedulers (List[ParamScheduler]) – list/tuple of parameter schedulers.

• names (Optional[List[str]]) – list of names of schedulers.

• save_history (bool) – whether to save history or not.

Examples

optimizer = torch.optim.SGD(
    [
        {"params": default_model.base.parameters(), "lr": 0.001},
        {"params": default_model.fc.parameters(), "lr": 0.01},
    ]
)

# CosineAnnealing increases the learning rate from 0.0 to 1.0
# over a cycle of 4 iterations
scheduler_1 = CosineAnnealingScheduler(optimizer, "lr", 0.0, 1.0, 4, param_group_index=0)

# CosineAnnealing increases the learning rate from 0.0 to 0.1
# over a cycle of 4 iterations
scheduler_2 = CosineAnnealingScheduler(optimizer, "lr", 0.0, 0.1, 4, param_group_index=1)

scheduler = ParamGroupScheduler(schedulers=[scheduler_1, scheduler_2],
                                names=["lr (base)", "lr (fc)"])

default_trainer.add_event_handler(Events.ITERATION_STARTED, scheduler)

@default_trainer.on(Events.ITERATION_COMPLETED)
def print_lr():
    print(optimizer.param_groups[0]["lr"],
          optimizer.param_groups[1]["lr"])

default_trainer.run([0] * 8, max_epochs=1)

0.0 0.0
0.1464... 0.01464...
0.4999... 0.04999...
0.8535... 0.08535...
...

New in version 0.4.5.

Methods

load_state_dict	Copies parameters from state_dict into this ParamScheduler.
simulate_values	Method to simulate scheduled values during num_events events.
state_dict	Returns a dictionary containing a whole state of ParamGroupScheduler.

load_state_dict(state_dict)

Copies parameters from state_dict into this ParamScheduler.

Parameters

state_dict (Mapping) – a dict containing parameters.

Return type

None

classmethod simulate_values(num_events, schedulers, **kwargs)

Method to simulate scheduled values during num_events events.

Parameters

• num_events (int) – number of events during the simulation.

• schedulers (List[_LRScheduler]) – lr_scheduler object to wrap.

• kwargs (Any) – kwargs passed to construct an instance of ignite.handlers.param_scheduler.ParamGroupScheduler.

Returns

event_index, value

Return type

List[List[int]]

state_dict()

Returns a dictionary containing a whole state of ParamGroupScheduler.

Returns

a dictionary containing a whole state of ParamGroupScheduler

Return type

dict