PiecewiseLinear#
- class ignite.handlers.param_scheduler.PiecewiseLinear(optimizer, param_name, milestones_values, save_history=False, param_group_index=None)[source]#
Piecewise linear parameter scheduler
- Parameters
optimizer (Optimizer) – torch optimizer or any object with attribute
param_groupsas a sequence.param_name (str) – name of optimizer’s parameter to update.
milestones_values (List[Tuple[int, float]]) – list of tuples (event index, parameter value) represents milestones and parameter. Milestones should be increasing integers.
save_history (bool) – whether to log the parameter values to engine.state.param_history, (default=False).
param_group_index (Optional[int]) – optimizer’s parameters group to use.
scheduler = PiecewiseLinear(optimizer, "lr", milestones_values=[(10, 0.5), (20, 0.45), (21, 0.3), (30, 0.1), (40, 0.1)]) # Attach to the trainer trainer.add_event_handler(Events.ITERATION_STARTED, scheduler) # # Sets the learning rate to 0.5 over the first 10 iterations, then decreases linearly from 0.5 to 0.45 between # 10th and 20th iterations. Next there is a jump to 0.3 at the 21st iteration and LR decreases linearly # from 0.3 to 0.1 between 21st and 30th iterations and remains 0.1 until the end of the iterations.
Examples
milestones_values = [(1, 1.0), (3, 0.8), (5, 0.2)] scheduler = PiecewiseLinear( default_optimizer, "lr", milestones_values=milestones_values) # Sets lr equal to 1 for till the first iteration # Then linearly reduces lr from 1 to 0.8 till the third iteration # Then linearly reduces lr from 0.8 to 0.5 till the fifth iteration default_trainer.add_event_handler(Events.ITERATION_STARTED, scheduler) @default_trainer.on(Events.ITERATION_COMPLETED) def print_lr(): print(default_optimizer.param_groups[0]["lr"]) default_trainer.run([0] * 6, max_epochs=1)
1.0 1.0 0.9 0.8 0.5 0.2
optimizer = torch.optim.SGD( [ {"params": default_model.base.parameters(), "lr": 0.1}, {"params": default_model.fc.parameters(), "lr": 1.0}, ] ) milestones_values1 = [(1, 0.1), (3, 0.08), (5, 0.02)] scheduler2 = PiecewiseLinear( optimizer, "lr", milestones_values=milestones_values1, param_group_index=0) # Sets lr equal to 0.1 for till the first iteration # Then linearly reduces lr from 0.1 to 0.08 till the third iteration # Then linearly reduces lr from 0.08 to 0.05 till the fifth iteration milestones_values2 = [(1, 1.0), (3, 0.8), (5, 0.2)] scheduler1 = PiecewiseLinear( optimizer, "lr", milestones_values=milestones_values2, param_group_index=1) # Sets lr equal to 1 for till the first iteration # Then linearly reduces lr from 1 to 0.8 till the third iteration # Then linearly reduces lr from 0.8 to 0.5 till the fifth iteration default_trainer.add_event_handler(Events.ITERATION_STARTED, scheduler1) default_trainer.add_event_handler(Events.ITERATION_STARTED, scheduler2) @default_trainer.on(Events.ITERATION_COMPLETED) def print_lr(): print(optimizer.param_groups[0]["lr"], optimizer.param_groups[1]["lr"]) default_trainer.run([0] * 6, max_epochs=1)
0.1 1.0 0.1 1.0 0.09 0.9 0.08 0.8 0.05 0.5 0.02 0.2
New in version 0.4.5.
Methods
Method to get current parameter values