Backtest IndexError at the right calendar boundary (get_step_time peeks calendar[index+1])

[zhaow-de]

🐛 Bug Description

TradeCalendarManager.get_step_time returns each trading step's interval as
(calendar[i], epsilon_change(calendar[i + 1])) — it reads the next bar to form the right
endpoint:

# qlib/backtest/utils.py:130-131  (TradeCalendarManager.get_step_time)
calendar_index = self.start_index + trade_step - shift
return self._calendar[calendar_index], epsilon_change(self._calendar[calendar_index + 1])

For the last trading step, calendar_index == self.end_index. If end_time is the dataset's
final calendar bar (end_index == len(self._calendar) - 1), then calendar_index + 1 is out
of bounds and the backtest dies with an opaque:

IndexError: index <N> is out of bounds for axis 0 with size <N>
  File ".../qlib/backtest/utils.py", line 131, in get_step_time
    return self._calendar[calendar_index], epsilon_change(self._calendar[calendar_index + 1])
  File ".../qlib/backtest/backtest.py", in collect_data_loop
    _trade_decision = trade_strategy.generate_trade_decision(_execute_result)
  File ".../qlib/contrib/strategy/signal_strategy.py", in generate_trade_decision
    trade_start_time, trade_end_time = self.trade_calendar.get_step_time(trade_step)

The index N out of bounds for size N form is the tell: the step needs calendar[end_index + 1]
but the calendar has no bar after end_time.

Root cause

The right-endpoint peek assumes there is always at least one bar after end_time. The
calendar is built with future=True:

# qlib/backtest/utils.py:69  (TradeCalendarManager.reset)
_calendar = Cal.calendar(freq=freq, future=True)

future=True is meant to supply a calendar that extends beyond the data so the +1 peek is safe.
But when a dataset has no future calendar configured (the common case for a self-built provider),
future=True silently falls back to the current calendar with only a warning:

WARNING qlib.data [data.py] - load calendar error: freq=day, future=True; return current calendar!
WARNING qlib.data [data.py] - You can get future calendar by referring to ...

So there is no bar after the last data day, and any backtest whose end_time is that last day
crashes at the final step. The failure is opaque (an IndexError deep in get_step_time, far from
the user's backtest(end_time=...) call) and easy to hit (it's simply "backtest through the end of
my data").

To Reproduce

1. Use a provider whose calendar has no future extension (a self-built qlib dataset; Cal.calendar(future=True) then warns and returns the current calendar).
2. Call qlib.backtest.backtest(start_time=..., end_time=<the last calendar day>, strategy=..., executor={"class": "SimulatorExecutor", ...}).
3. The run dies at the final step with IndexError: index N is out of bounds for axis 0 with size N in TradeCalendarManager.get_step_time (utils.py:131).

(Ending one or more bars before the last calendar day does not crash, because calendar[index+1] then exists.)

Expected Behavior

A backtest ending on the last available calendar bar should either succeed or fail with a clear,
actionable message — not an opaque IndexError. Options:

• In get_step_time, clamp the right endpoint at the boundary, e.g.
  right = epsilon_change(self._calendar[i + 1]) if i + 1 < len(self._calendar) else <end-of-last-bar>
  (mirroring the day-end logic already used elsewhere, e.g. day_start + pd.Timedelta(days=1) at
  utils.py:154), so the final step's interval is well-defined without a future bar.
• And/or validate up front in backtest() that end_time leaves at least one trailing bar when no
  future calendar is available, raising a clear error (or auto-clamping) instead of crashing mid-loop.
• And/or promote the future=True fallback warning to a clearer, louder signal that the right
  boundary is unsafe for backtesting through the data end.

Environment

• Qlib version: 0.9.7 (qlib/backtest/utils.py:131, TradeCalendarManager.get_step_time)
• Python version: 3.12
• freq: day; provider: a self-built daily dataset with no future calendar.

Additional context

Discovered while building an out-of-sample walk-forward harness whose last window's end_time was
the dataset's final calendar day. Worked around it by capping end_time a couple of bars before
the calendar end (so calendar[index+1] always exists). A boundary-safe get_step_time (or an
upfront check/clamp in backtest) would remove the need for that workaround and turn a cryptic
IndexError into correct behavior or a clear error.

[zhaow-de]

Thanks for laying out shared acceptance criteria, @hyperxiaoerxz-hash!

I've updated my PR (commit fd54ac70) so each invariant is demonstrated by a committed, red-verified test in tests/backtest/test_calendar_boundary.py. Mapping below:

#	Invariant	How it's addressed
1	get_step_time must not raise IndexError on the final bar	Guarded peek: when calendar_index + 1 == len(self._calendar), fall back to the end of the current bar's period. test_get_step_time_at_last_calendar_bar (real future calendar) + test_boundary_fallback_day.
2	Non-boundary right endpoint stays epsilon_change(calendar[index + 1])	The calendar_index + 1 < len branch is the original expression, untouched. test_non_boundary_step_unchanged.
3	Pre-boundary backtests produce identical step intervals	test_pre_boundary_steps_identical_to_old_peek asserts every step equals (calendar[i], epsilon_change(calendar[i+1])); tests/test_all_pipeline.py (whose end_time is well before the boundary) is unchanged and still passes.
4	Fallback covered for day and an intraday freq	test_boundary_fallback_day and test_boundary_fallback_minute. The fallback uses left + Freq.get_timedelta(*Freq.parse(self.freq)), so it's granularity-agnostic.
5	shift tested (it moves the effective boundary)	test_shift_moves_effective_boundary: shift=0 at the last step takes the fallback; shift=1 moves calendar_index back one bar so the normal peek path applies and returns epsilon_change(calendar[-1]). (All call sites use shift of 0 or 1.)
6	trade_step=None / current-step path covered	test_current_step_path advances trade_step to the final step and calls get_step_time() with no argument.
7	finished() stays correct — no extra/lost iteration	test_full_loop_finished_and_step_count runs the #1063 loop shape and asserts exactly trade_len steps with finished() true afterward. The fix touches only the endpoint computation, not step()/finished()/trade_len.
8	epsilon_change applied consistently to the fallback	epsilon_change is applied once to whichever right is selected (return left, epsilon_change(right)), so both paths share the closed-interval convention.
9	Cover the same-shape report in #1063	Same IndexError and root cause; this PR now closes #1063 as well. The fallback removes the need for the "set end_time before the last calendar date" workaround.

Two implementation notes:

• The fallback mirrors the existing day-end idiom in get_data_cal_range (utils.py:154, day_start + Timedelta(days=1)), generalized across frequencies via Freq.
• Each boundary-touching test was confirmed to fail with the original IndexError on the pre-fix code and to pass after the fix; the non-boundary/pre-boundary tests pass on both, so they can't silently regress.