Skip to content

Fix ReduceLROnPlateauScheduler ignoring min_lr - #3876

Open
G26karthik wants to merge 4 commits into
pytorch:masterfrom
G26karthik:fix/reduce-lr-on-plateau-min-lr
Open

G26karthik wants to merge 4 commits into
pytorch:masterfrom
G26karthik:fix/reduce-lr-on-plateau-min-lr

Conversation

@G26karthik

@G26karthik G26karthik commented Oct 7, 2026 •

Copy link
Copy Markdown

Fixes #3875

Description:

ReduceLROnPlateauScheduler replaced min_lr with 0 when param_group_index is None, and with param_group_index=k > 0 it put the floor at index k while _reduce_lr reads min_lrs[0] for the single selected group. In both cases the LR decayed past min_lr.

This PR stops rewriting min_lr and passes it to torch's ReduceLROnPlateau as given. torch expands a float to one value per param group, so _reduce_lr reads the right floor whether one group or all groups are scheduled. The TypeError for a non-float min_lr together with param_group_index stays. A list min_lr without param_group_index now reaches torch too, instead of being replaced by 0.

For the test, I parametrized the existing test_reduce_lr_on_plateau_scheduler over param_group_index (None, 0, 1) and raised its min_lr from 1e-7 to 0.3, so the last reduction now hits the floor (0.25 is clamped to 0.3). The old 1e-7 was never reached, which is why the test didn't catch this. On master the None and 1 cases fail (they end at 0.25), and with the fix all 6 cases pass. The rest of tests/ignite/handlers/test_param_scheduler.py passes locally (65 passed; test_simulate_and_plot_values needs matplotlib, which isn't in my env), and ruff check/format are clean.

Check list:

  • New tests are added (if a new feature is added)
  • New doc strings: description and/or example code are in RST format
  • Documentation is updated (if required)

min_lr was replaced by 0 when param_group_index is None, and with
param_group_index > 0 the floor was stored at an index _reduce_lr never
reads. Pass min_lr to torch's ReduceLROnPlateau unchanged; torch expands
a float to every param group.
Copilot AI balanced review requested due to automatic review settings October 7, 2026 03:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added the module: handlers Core Handlers module label Oct 7, 2026
@aaishwarymishra

Copy link
Copy Markdown
Collaborator

Could we extend this test with a few additional plateau metrics after the LR reaches 0.3, and assert that it remains at 0.3 on subsequent evaluations? Also, for param_group_index=None, the test currently checks only the first group asserting the full LR history would verify that both groups respect min_lr.?

@G26karthik

Copy link
Copy Markdown
Author

Thanks for the review! Done in d3668ab. The test now runs four more plateau evaluations after the LR reaches 0.3 and checks that it stays there, and it compares the full LR history of every param group the scheduler updates, so with param_group_index=None both groups are checked at every step.

@G26karthik

Copy link
Copy Markdown
Author

Hi @vfdev-5, sorry to bother you! This one was approved by @aaishwarymishra on Oct 7 and CI is green. Could you take a look when you get a chance? #3880 is a related fix for the same scheduler (its state_dict can't be restored from a checkpoint), in case you'd like to review them together. Thanks!

Comment thread ignite/handlers/param_scheduler.py Outdated
min_lr = scheduler_kwargs.get("min_lr", 0.0)
if param_group_index is not None and not isinstance(min_lr, float):
raise TypeError(f"When param_group_index is given, min_lr should be a float, but given {type(min_lr)}")
_scheduler_kwargs = scheduler_kwargs.copy()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need to make a copy now?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, good point. It was only there so overwriting min_lr wouldn't touch the caller's dict, and **scheduler_kwargs is already a fresh dict, so setting verbose on it is fine too. Removed it in 9832774.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug module: handlers Core Handlers module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ReduceLROnPlateauScheduler ignores min_lr unless param_group_index is 0

4 participants