Skip to content

Allow subscriptions to declare their real bar period - #9637

Open
YadavKapil wants to merge 1 commit into
QuantConnect:masterfrom
YadavKapil:feature/subscription-bar-period
Open

Allow subscriptions to declare their real bar period#9637
YadavKapil wants to merge 1 commit into
QuantConnect:masterfrom
YadavKapil:feature/subscription-bar-period

Conversation

@YadavKapil

Copy link
Copy Markdown
Contributor

Description

LEAN derives a bar's length from the name of its resolution, so data whose stored period differs from the nominal one is misdescribed. Concretely, 30-minute bars kept in the minute folders are treated as 1-minute bars: FillForwardEnumerator manufactures ~29 synthetic bars per real bar, and every bar is stamped with an end time 29 minutes too early.

This adds an optional barPeriod to SubscriptionDataConfig, surfaced on AddEquity, AddIndex, AddOption, AddIndexOption, AddSecurity and UniverseSettings:

AddEquity("SPY", Resolution.Minute, barPeriod: TimeSpan.FromMinutes(30));

It defaults to null, so the period continues to be derived from the resolution and no existing algorithm changes behaviour. Resolution is deliberately untouched, so data still resolves from the same files and no data needs regenerating.

With the period declared, ShouldFillForward sees a gap equal to the period rather than 30x it, so no synthetic bars are emitted when the data is complete, and genuine gaps are still filled with exactly one bar.

Four latent Resolution-as-duration reads

Four places read Resolution where they mean a duration. They agree today because Increment is a pure function of Resolution, so each change is provably equivalent for any subscription that does not declare a period, and the PR includes tests asserting that equivalence across all five resolutions.

Location Issue
SubscriptionCollection Derived the fill-forward cadence from the resolution enum, which would reinstate the synthetic bars
FillModel.ShouldWaitForFreshData Tested Resolution.Hour/Daily to decide whether a resting order fills at the bar open rather than its close. Now Increment > Time.OneMinute, which selects exactly Hour and Daily. Without this, a declared 30-minute bar would fill at the close of the following bar
FillModel stale threshold Sized via GetHighestResolution().ToTimeSpan(). Added GetHighestResolutionSpan(), taking the minimum Increment
CreateConsolidator Compared a TimeSpan against Increment and a Resolution against Resolution in one expression. Both now compare durations

The consolidator change also fixes a pre-existing bug: against coarser-than-nominal data, SMA(symbol, 20, Resolution.Minute) evaluated Minute < Minute as false, skipped the guard, matched the identity branch and silently returned the coarser bars labelled as minute bars. It now throws.

Equals/GetHashCode include the declared period so configs differing only by it are not deduplicated, and the copy constructor drops it when the resolution changes, since it described the previous resolution.

Testing

New fixture Tests/Common/Data/SubscriptionDataConfigBarPeriodTests.cs (23 tests) covers the equivalence proofs, config identity, copy-constructor inheritance, and validation. Two new FillForwardEnumeratorTests assert that a declared period emits zero synthetic bars with correct end times, while a genuine gap is still filled.

Suites run green in quantconnect/lean:foundation:

Filter Result
SubscriptionDataConfigBarPeriodTests 23 passed
FillForwardEnumeratorTests, SubscriptionCollectionTests, SubscriptionDataConfigTests 315 passed
Fills, FillModel 544 passed, 2 skipped
Consolidator, HistoryRequest, UniverseSettings 669 passed

Full solution builds clean on net10.0.

LEAN derives a bar's length from the name of its resolution, so data whose
stored period differs from the nominal one is misdescribed. Thirty minute bars
kept in the minute folders are treated as one minute bars, which makes the fill
forward enumerator manufacture 29 synthetic bars per real bar and stamps every
bar with an end time 29 minutes too early.

Add an optional barPeriod to SubscriptionDataConfig, surfaced on AddEquity,
AddIndex, AddOption, AddIndexOption, AddSecurity and UniverseSettings. It
defaults to null, so the period continues to be derived from the resolution and
no existing algorithm changes behaviour. Resolution is untouched, so data is
still resolved from the same files and no data needs regenerating.

Four places read Resolution where they mean a duration. They agree today because
Increment is a pure function of Resolution, so each is equivalent for any
subscription that does not declare a period:

- SubscriptionCollection derived the fill forward cadence from the resolution
  enum, which would reinstate the synthetic bars.
- FillModel.ShouldWaitForFreshData tested Resolution.Hour/Daily to decide whether
  a resting order fills at the bar open rather than its close. Expressed as
  Increment > Time.OneMinute, which selects exactly Hour and Daily. Without this
  a declared thirty minute bar would fill at the close of the following bar.
- FillModel sized the stale data threshold with GetHighestResolution().ToTimeSpan().
  Added GetHighestResolutionSpan, which takes the minimum Increment.
- CreateConsolidator compared a TimeSpan against Increment and a Resolution
  against Resolution in one expression. Both now compare durations, which also
  stops a minute consolidator being silently created as an identity consolidator
  over coarser data.

SubscriptionDataConfig.Equals and GetHashCode include the declared period so
configs differing only by it are not deduplicated, and the copy constructor drops
it when the resolution changes, since it described the previous resolution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1947964c-36cc-4afb-82a0-9307f00731d8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant