Skip to content

pkg/workflows/dontime: introduce feature flagged fix for sequence number inconsistency - #2361

Draft
jmank88 wants to merge 3 commits into
mainfrom
CRE-5838-dontime-seqnums
Draft

pkg/workflows/dontime: introduce feature flagged fix for sequence number inconsistency#2361
jmank88 wants to merge 3 commits into
mainfrom
CRE-5838-dontime-seqnums

Conversation

@jmank88

@jmank88 jmank88 commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

✅ API Diff Results - github.com/smartcontractkit/chainlink-common

✅ Compatible Changes (6)

pkg/settings/cresettings.Schema (1)
  • DonTimeSequencedTimestampsEnabled — ➕ Added
pkg/workflows/dontime.(*Factory) (1)
  • InitLimits — ➕ Added
pkg/workflows/dontime/pb.(*ObservedDonTimes) (3)
  • EarliestTS — ➕ Added

  • GetTimestampsBySequence — ➕ Added

  • MaxSeqNum — ➕ Added

pkg/workflows/dontime/pb.ObservedDonTimes (1)
  • TimestampsBySequence — ➕ Added

📄 View full apidiff report

@jmank88
jmank88 force-pushed the CRE-5838-dontime-seqnums branch from 01f63a7 to 9cc0558 Compare September 8, 2026 22:14
Comment thread pkg/workflows/dontime/plugin.go Outdated
@jmank88
jmank88 requested a review from bolekk September 9, 2026 19:28
@@ -0,0 +1,24 @@
package pb

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why in pb?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seemed fitting as a method, but could be a helper func instead 🤷

Comment thread pkg/workflows/dontime/plugin.go Outdated
Comment thread pkg/workflows/dontime/plugin.go
WorkflowExecutionID: req.WorkflowExecutionID,
SeqNum: req.SeqNum,
Timestamp: 0,
Err: fmt.Errorf("requested seqNum %d for executionID %s is greater than expected based on the max seqNum observed so far %d",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should error here. What if the plugin didn't process all lower seq numbers in the previous round? I thought we would always add all local requests to observation.

timestampNodePairs = append(timestampNodePairs, timestampNodePair{Timestamp: observation.Timestamp, NodeID: idx})
outcome := prevOutcome

// Compare with prior outcome to ensure DON time never goes backward.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this check happen earlier, before we go into un/sequencedOutcome() ?


// sequencedOutcome executed the updated outcome logic to produce a sequenced map of [pb.ObservedDonTimes.TimestampsBySequence].
func (p *Plugin) sequencedOutcome(ctx context.Context, outctx ocr3types.OutcomeContext, _ types.Query, aos []types.AttributedObservation, donTime int64) (ocr3types.Outcome, error) {
observationCounts := map[string]int64{} // counts how many nodes reported where a new DON timestamp might be needed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: comment what is the key in this map

continue
}

if !observation.GetLimitByBatchSizeFlag() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm I should've cleaned that up. We can assume this is enabled everywhere by now

observationCounts[id]++
} else if requestSeqNum > currSeqNum {
// This should never happen since we don't include out of sequence requests in the Observation phase
p.lggr.Errorf("request seqNum %d for executionID %s is greater than the current seqNum %d",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's an error any more. Shouldn't we just process the highest number that we can reach consensus on and allow for gaps? cc @cedric-cordenier if I'm missing something here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants