Skip to content

An exception raised while handling an intercepted request is reported against a later, unrelated test #43

Description

@ker0x

Environment: playwright-symfony dev-main (270ff6fe), playwright-php/playwright v1.4.0, PHP 8.5.9, Symfony 8.1, Chromium headless.

What happens

Intercepted requests are handled from the route event, which the transport drains lazily:

PlaywrightKernelClient::handleInternalRequest()   (src/Client/PlaywrightKernelClient.php:626)
  ← PlaywrightKernelClient route handler          (src/Client/PlaywrightKernelClient.php:546)
  ← Event\EventEmitter::emit()                    (playwright/src/Event/EventEmitter.php:60)
  ← Page\PageEventHandler                         (playwright/src/Page/PageEventHandler.php:27)
  ← JsonRpcTransport::processEvents()

If handling throws, the exception is raised from whichever Playwright call happens to be draining events at that moment — not from the call that triggered the request. In practice that is a later statement, frequently in a later test, because the browser keeps issuing sub-resource requests after the statement that navigated.

Observed in a suite of 8 test classes: one page referencing a missing image produced

✘ Carousel pages through gallery images      NotFoundHttpException: Not Found
✘ Grid list toggle switches layout …         NotFoundHttpException: Not Found
✘ Clicking a card navigates …                NotFoundHttpException: Not Found
✘ Hovering a card highlights its marker      NotFoundHttpException: Not Found

— four failures with identical stacks, only the first of which had anything to do with the cause. In another run the same class of error was reported against tearDownAfterClass():

These after-last-test methods errored:
1) …\CreateGuestSessionTest::tearDownAfterClass
   UnprocessableEntityHttpException: Form validation failed

The stack always points at the application controller, so the reported test looks guilty. Tests that pass in isolation fail in a suite, and the first red test is usually not the broken one.

Why it matters

This is a diagnosis problem rather than a correctness one, but it is expensive: it presents as flakiness and as suite-order dependence. While porting an existing suite I twice pursued the wrong root cause because the failure was attributed to an innocent test, and once concluded a change of mine had introduced flakiness when it had not.

Suggested fix

Some options, roughly in order of how much they'd help:

  1. Drain pending route events at test teardown, so an in-flight failure is attributed to the test that produced it rather than the next one.
  2. Wrap the exception with the request that caused it — e.g. RequestHandlingFailed: GET /image/foo.jpg failed while handling an intercepted request, keeping the original as previous. Even without changing attribution, this alone would make the reports actionable.
  3. Record failures and surface them at a deterministic point (end of the triggering test) instead of re-throwing from an arbitrary transport call.

Note this interacts with (but is separate from) the catch: false issue: handling requests with $catch = true removes the most common source of these exceptions, but any exception escaping the bridge itself would still be misattributed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions