Allow subscriptions to declare their real bar period - #9637
Open
YadavKapil wants to merge 1 commit into
Open
Conversation
LEAN derives a bar's length from the name of its resolution, so data whose stored period differs from the nominal one is misdescribed. Thirty minute bars kept in the minute folders are treated as one minute bars, which makes the fill forward enumerator manufacture 29 synthetic bars per real bar and stamps every bar with an end time 29 minutes too early. Add an optional barPeriod to SubscriptionDataConfig, surfaced on AddEquity, AddIndex, AddOption, AddIndexOption, AddSecurity and UniverseSettings. It defaults to null, so the period continues to be derived from the resolution and no existing algorithm changes behaviour. Resolution is untouched, so data is still resolved from the same files and no data needs regenerating. Four places read Resolution where they mean a duration. They agree today because Increment is a pure function of Resolution, so each is equivalent for any subscription that does not declare a period: - SubscriptionCollection derived the fill forward cadence from the resolution enum, which would reinstate the synthetic bars. - FillModel.ShouldWaitForFreshData tested Resolution.Hour/Daily to decide whether a resting order fills at the bar open rather than its close. Expressed as Increment > Time.OneMinute, which selects exactly Hour and Daily. Without this a declared thirty minute bar would fill at the close of the following bar. - FillModel sized the stale data threshold with GetHighestResolution().ToTimeSpan(). Added GetHighestResolutionSpan, which takes the minimum Increment. - CreateConsolidator compared a TimeSpan against Increment and a Resolution against Resolution in one expression. Both now compare durations, which also stops a minute consolidator being silently created as an identity consolidator over coarser data. SubscriptionDataConfig.Equals and GetHashCode include the declared period so configs differing only by it are not deduplicated, and the copy constructor drops it when the resolution changes, since it described the previous resolution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1947964c-36cc-4afb-82a0-9307f00731d8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
LEAN derives a bar's length from the name of its resolution, so data whose stored period differs from the nominal one is misdescribed. Concretely, 30-minute bars kept in the
minutefolders are treated as 1-minute bars:FillForwardEnumeratormanufactures ~29 synthetic bars per real bar, and every bar is stamped with an end time 29 minutes too early.This adds an optional
barPeriodtoSubscriptionDataConfig, surfaced onAddEquity,AddIndex,AddOption,AddIndexOption,AddSecurityandUniverseSettings:It defaults to
null, so the period continues to be derived from the resolution and no existing algorithm changes behaviour.Resolutionis deliberately untouched, so data still resolves from the same files and no data needs regenerating.With the period declared,
ShouldFillForwardsees a gap equal to the period rather than 30x it, so no synthetic bars are emitted when the data is complete, and genuine gaps are still filled with exactly one bar.Four latent
Resolution-as-duration readsFour places read
Resolutionwhere they mean a duration. They agree today becauseIncrementis a pure function ofResolution, so each change is provably equivalent for any subscription that does not declare a period, and the PR includes tests asserting that equivalence across all five resolutions.SubscriptionCollectionFillModel.ShouldWaitForFreshDataResolution.Hour/Dailyto decide whether a resting order fills at the bar open rather than its close. NowIncrement > Time.OneMinute, which selects exactlyHourandDaily. Without this, a declared 30-minute bar would fill at the close of the following barFillModelstale thresholdGetHighestResolution().ToTimeSpan(). AddedGetHighestResolutionSpan(), taking the minimumIncrementCreateConsolidatorTimeSpanagainstIncrementand aResolutionagainstResolutionin one expression. Both now compare durationsThe consolidator change also fixes a pre-existing bug: against coarser-than-nominal data,
SMA(symbol, 20, Resolution.Minute)evaluatedMinute < Minuteas false, skipped the guard, matched the identity branch and silently returned the coarser bars labelled as minute bars. It now throws.Equals/GetHashCodeinclude the declared period so configs differing only by it are not deduplicated, and the copy constructor drops it when the resolution changes, since it described the previous resolution.Testing
New fixture
Tests/Common/Data/SubscriptionDataConfigBarPeriodTests.cs(23 tests) covers the equivalence proofs, config identity, copy-constructor inheritance, and validation. Two newFillForwardEnumeratorTestsassert that a declared period emits zero synthetic bars with correct end times, while a genuine gap is still filled.Suites run green in
quantconnect/lean:foundation:SubscriptionDataConfigBarPeriodTestsFillForwardEnumeratorTests,SubscriptionCollectionTests,SubscriptionDataConfigTestsFills,FillModelConsolidator,HistoryRequest,UniverseSettingsFull solution builds clean on net10.0.