Skip to content

Firefox browser lane is findings-only (required: false) while the justfile documents it as required #189

Description

@lannbot

Summary

The Firefox browser lane is a findings-only lane (required: false), so a
Firefox-only regression exits 0 and leaves the post-merge browser job green.
The justfile documents the same lane as required, and the README's parity
claim rests on it. The flag and the prose disagree; one of them is wrong.

Details

The expectation sets the lane non-gating —
harness/browser/expectations/firefox.ts:41:

export const firefox: LaneExpectation = {
  lane: "firefox",
  required: false,

and firefox.ts:46-47 restates the intent:

  // Findings lane: totals are recorded for drift detection but the driver
  // does not gate on them (`required: false`).

The driver keys its exit status on that flag — tools/browser/run-lane.ts:287:

  Deno.exit(exp.required ? 1 : 0);

The contradicting comment sits directly above the recipe — justfile:182-185:

# chromium and firefox are required (chromium expects exact Deno-lane
# parity; the firefox driver sets the JSPI pref itself — shipped-channel
# config, unlike the jsshell); webkit is best-effort per
# docs/architecture.md §3/§12 (issue #11).

CI runs the lane through the gating _step (WebKit uses
_step-tolerated) — .github/justfile:72-77:

    @just gha::_step "browser-lane chromium"
    @just gha::_step "browser-lane firefox"
    ...
    @just gha::_step-tolerated "browser-lane webkit"

The gating wrapper is therefore correct in intent but inert in effect: with
required: false, unexpected failures and totals drift print deviations and
exit 0.

README.md:22 also leans on the lane:

identical on Deno, Chromium, and Firefox (behind its JSPI pref)

Decision needed

Pick one and record the rationale:

  • Flip required to true — the lane really is meant to gate, and the
    justfile comment plus .github/justfile's use of _step already assume so.
  • Keep required: false — e.g. because JSPI is behind a pref and thus not
    shipped-by-default in Firefox, making the lane inherently advisory. Then the
    justfile:182-185 comment must stop calling firefox required, and the
    README parity claim should be reworded to say the Firefox result is recorded,
    not enforced.

Either way, the codebase should not carry both statements.

Provenance

Found in structured audit round 3 (2026-08-21). Audit/testing regime: #175.

Metadata

Metadata

Assignees

No one assigned

    Labels

    p2Minor bugs; desirable lower-priority featurespolicyDecision owed on deltic-owned semanticstestingTest coverage / verification infrastructure

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions