Skip to content

build: add lockfile-versioned core telemetry PR validation - #336863

Closed
Joaquín Ruales (jruales) wants to merge 1 commit into
mainfrom
jruales-telemetry-pr-gate
Closed

Joaquín Ruales (jruales) wants to merge 1 commit into
mainfrom
jruales-telemetry-pr-gate

Conversation

@jruales

Copy link
Copy Markdown
Contributor

Summary

Add an Azure-equivalent core telemetry validation step to the existing Check metadata job, using the exact extractor version recorded in the root package-lock.json (currently 1.20.5). This is a prevention-only follow-up to #336857; it does not change production telemetry or replace the event fix.

The new step runs the standalone extractor through npx, with --sourceDir . --excludedDir extensions --applyEndpoints and a 4096 MiB Node heap, matching Azure's core extraction scope and options without installing VS Code's native dependencies. It adds no independent version pin. Workflow permissions, PR branch filters, checkout, Node setup, and failure propagation are unchanged.

Why this escaped PR validation

#336747 upgraded the root extractor dependency and lockfile from 1.20.4 to 1.20.5, but .github/workflows/telemetry.yml independently pinned 1.14.0. The green metadata job for #336704 explicitly ran that old version. Azure used the repository-installed extractor and rejected the event-specific agentHost.firstResponse.sessionId because it collides case-insensitively with common sessionID (representative failure).

The existing 1.14.0 all-repository metadata check remains verbatim, retaining its current extension coverage. Simply upgrading that mixed-namespace invocation to 1.20.5 introduces cross-namespace conflicts between core and extensions, as well as exposing unrelated extension errors. This narrowly scoped PR adds the reliable core gate rather than removing existing coverage, changing extension telemetry, or weakening validation. It deliberately does not claim that every existing telemetry check now uses one version.

Validation

Executed the exact new run block read from the workflow under Bash with failure propagation, using extractor 1.20.5 and the configured 4096 MiB heap. The controlled snapshot contained all 12,396 tracked non-extension files, not just the affected handler directory.

Source snapshot Result
Original source at ff61d31952aa3ca982afe61db10f530d5db0b38e Exit 1, with only Property sessionid on event agentHost.firstResponse is overloaded by a common property
Same snapshot with only #336857's two production-file changes Exit 0, extracting 484 events

The corrected snapshot files were compared byte-for-byte with commit 2d69870de14492757ed6187ce698ee10e2b4f45d. Output assertions confirmed that common sessionid remains, event agentsessionid exists, and event-specific sessionid is absent. Those event changes exist only in the temporary validation snapshot, not in this PR.

Also verified that removing the new step yields exactly the original parsed workflow, and checked YAML parsing, shell syntax, and diff whitespace. Inspection and a real-source control run confirmed that 1.20.5 checks collisions without --applyEndpoints; the flag is retained for Azure parity, not as the switch enabling collision validation. No product build or Azure pipeline was queued.

Relationship to the event fix

#336857 has now merged into main as 305099e10f60b43b5d6eb57b532b95108d09e092. Before that fix (or an equivalent correction), this new core check is expected to fail on the existing collision, as the red validation demonstrates. The gate prevents recurrence; the separate event fix repairs the invalid metadata. This draft changes only .github/workflows/telemetry.yml.

Keep the existing metadata check and add an Azure-equivalent core check using the extractor version from package-lock.json and a 4096 MiB heap. This catches event properties that collide with common telemetry properties before merge.

Prevention follow-up to #336857; no production event changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 18, 2026 23:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The focused workflow change correctly matches the lockfile version and Azure core extraction configuration.

Review effort: Balanced
Findings: None

What changed in this PR

Adds lockfile-versioned core telemetry validation to catch metadata conflicts during PR checks.

Changes:

  • Resolves the extractor version from package-lock.json.
  • Validates core telemetry using Azure-equivalent scope, options, and heap size.
File Description
.github/​workflows/​telemetry.yml Adds the core telemetry validation step.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

Screenshot Changes

Base: de3a59e0 Current: 133714be

Changed (2)

sessions/sessionChatInputToolbar/SessionChatPills_HorizontalOverflow/Dark
Before After
before after
sessions/sessionChatInputToolbar/SessionChatPills_HorizontalOverflow/Light
Before After
before after

2 insignificant change(s) omitted (≤20 px, Δ≤2). See CI logs for details.

@jruales

Copy link
Copy Markdown
Contributor Author

Closing for now as superseded by #336865, which addresses the core telemetry validation gap through the shared official extraction path and also aligns Copilot validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants