feat: Playwright E2E tests — 5 critical flows, CI workflow#133
feat: Playwright E2E tests — 5 critical flows, CI workflow#133TerrifiedBug merged 12 commits intomainfrom
Conversation
… shared components
- Add e2e/tsconfig.json with moduleResolution: node so Playwright can resolve the generated Prisma client and path aliases at runtime - Fix cleanup.ts: prisma.alertDelivery → prisma.deliveryAttempt to match the actual Prisma schema (model is DeliveryAttempt, not AlertDelivery) - src/generated/prisma must be generated via `npx prisma generate` before running E2E tests (added to CI workflow already)
Greptile SummaryThis PR introduces a solid Playwright E2E test suite with Page Object Model architecture, storageState-based auth, a Prisma seed/cleanup harness, and a CI workflow. The overall structure is well-thought-out — the project dependency chain, sequential execution, shared fixtures, and manual FK-ordered cleanup are all correct. Two bugs in Key findings:
Confidence Score: 4/5Two P1 bugs in global-setup.ts will reliably break CI; safe to merge once those two lines are fixed. The two P1 issues in global-setup.ts are genuine, reproducible defects on the changed path: the ENOENT crash prevents CI from running at all on a fresh checkout, and the permissive waitForURL glob means a failed login silently poisons the auth state for the entire run. Both are one-or-two-line fixes. All remaining findings are P2 style/completeness issues that don't block the test suite from working once the P1s are resolved. e2e/global-setup.ts requires two targeted fixes before the CI workflow will be reliable. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
CI[CI Workflow / pnpm test:e2e] --> GS[global-setup.ts\nsetup project]
GS --> DB[(Postgres\nService Container)]
GS --> SEED[seed.ts\nCreate: user · team · env\npipeline · nodes · fleet node\nalert rules + events]
SEED --> WRITE[fs.writeFile\ne2e/.auth/seed-result.json]
GS --> LOGIN[UI Login\n/login → click Sign In]
LOGIN --> WAIT["waitForURL(**/*)\n⚠ matches /login too"]
WAIT --> AUTH[storageState → e2e/.auth/user.json]
AUTH --> TESTS
subgraph TESTS[Chromium Project — sequential, workers=1]
direction TB
A[alerts.spec.ts] --> B[auth.spec.ts]
B --> C[deploy.spec.ts\nreads seed-result.json]
C --> D[fleet.spec.ts]
D --> E[pipeline-crud.spec.ts\ndeletes seeded pipeline]
end
TESTS --> GT[global-teardown.ts]
GT --> CLEAN[cleanup.ts\ndelete all test data]
CLEAN --> DB
Prompt To Fix All With AIThis is a comment left during a code review.
Path: e2e/global-setup.ts
Line: 36
Comment:
**`waitForURL("**/*")` matches the login page itself — failed auth is undetected**
`"**/*"` is a glob that matches every URL, including `http://localhost:3000/login`. Playwright's `waitForURL` resolves the moment the current URL matches the pattern. Because the page is already on `/login` when the click happens (and would stay there if credentials are wrong, the server returns an error, or NextAuth rejects the login), this line resolves immediately in those cases. The auth state is then saved to `e2e/.auth/user.json` with an unauthenticated session, causing every downstream test to fail with confusing "not authenticated" errors rather than a clear setup failure.
The fix is to wait for a URL that is specifically *not* the login page:
```suggestion
await page.waitForURL((url) => !url.pathname.includes("/login"), { timeout: 15_000 });
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: e2e/global-setup.ts
Line: 14-20
Comment:
**`e2e/.auth/` directory is never created — CI will throw ENOENT**
`e2e/.auth/` is listed in `.gitignore`, so it won't exist on a fresh clone or CI checkout. `fs.writeFile("e2e/.auth/seed-result.json", ...)` does **not** create missing parent directories; it throws `ENOENT: no such file or directory`. This crashes the global-setup before any test runs.
Playwright's own `storageState({ path })` call (line 38) handles directory creation internally, so that write is safe — but the manual `writeFile` here is not.
Add an `mkdir` before the write:
```suggestion
const fs = await import("fs/promises");
await fs.mkdir("e2e/.auth", { recursive: true });
await fs.writeFile(
"e2e/.auth/seed-result.json",
JSON.stringify(result, null, 2),
);
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: e2e/tests/deploy.spec.ts
Line: 26-37
Comment:
**Test name promises a deployment badge check that isn't performed**
`"should show deployment badge on pipeline list after deploy"` only asserts that the pipeline name appears in the list (`expectPipelineInList`). It never calls `pipelinesPage.expectDeploymentBadge("E2E Test Pipeline")`, which is the method that actually checks the badge (verifying the row no longer says "Draft"). The test as written will pass even if the pipeline never deployed successfully — it's just a duplicate of the visibility check already covered by the pipeline-crud tests.
Call `expectDeploymentBadge` to make the assertion match the test's stated intent:
```typescript
await pipelinesPage.expectPipelineInList("E2E Test Pipeline");
await pipelinesPage.expectDeploymentBadge("E2E Test Pipeline");
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: e2e/tests/auth.spec.ts
Line: 14-16
Comment:
**Credentials hardcoded instead of using `TEST_USER` constants**
`"e2e@test.local"` and `"TestPassword123!"` are repeated inline here (and again on lines 27, 40–41) rather than using the `TEST_USER` constants already defined in `e2e/helpers/constants.ts`. If the seed values ever change, these tests would silently diverge from the seed data.
```suggestion
await loginPage.login(TEST_USER.email, TEST_USER.password);
```
Import at the top of the file:
```typescript
import { TEST_USER } from "../helpers/constants";
```
This also applies to the other three credential literals in this file (lines 27, 40, 41).
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "Merge branch 'main' into feat/e2e-tests" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| await page.getByRole("button", { name: /sign in/i }).click(); | ||
|
|
||
| await page.waitForURL("**/*", { timeout: 15_000 }); | ||
|
|
There was a problem hiding this comment.
waitForURL("**/*") matches the login page itself — failed auth is undetected
"**/*" is a glob that matches every URL, including http://localhost:3000/login. Playwright's waitForURL resolves the moment the current URL matches the pattern. Because the page is already on /login when the click happens (and would stay there if credentials are wrong, the server returns an error, or NextAuth rejects the login), this line resolves immediately in those cases. The auth state is then saved to e2e/.auth/user.json with an unauthenticated session, causing every downstream test to fail with confusing "not authenticated" errors rather than a clear setup failure.
The fix is to wait for a URL that is specifically not the login page:
| await page.waitForURL((url) => !url.pathname.includes("/login"), { timeout: 15_000 }); |
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/global-setup.ts
Line: 36
Comment:
**`waitForURL("**/*")` matches the login page itself — failed auth is undetected**
`"**/*"` is a glob that matches every URL, including `http://localhost:3000/login`. Playwright's `waitForURL` resolves the moment the current URL matches the pattern. Because the page is already on `/login` when the click happens (and would stay there if credentials are wrong, the server returns an error, or NextAuth rejects the login), this line resolves immediately in those cases. The auth state is then saved to `e2e/.auth/user.json` with an unauthenticated session, causing every downstream test to fail with confusing "not authenticated" errors rather than a clear setup failure.
The fix is to wait for a URL that is specifically *not* the login page:
```suggestion
await page.waitForURL((url) => !url.pathname.includes("/login"), { timeout: 15_000 });
```
How can I resolve this? If you propose a fix, please make it concise.| const result = await seed(prisma); | ||
|
|
||
| const fs = await import("fs/promises"); | ||
| await fs.writeFile( | ||
| "e2e/.auth/seed-result.json", | ||
| JSON.stringify(result, null, 2), | ||
| ); |
There was a problem hiding this comment.
e2e/.auth/ directory is never created — CI will throw ENOENT
e2e/.auth/ is listed in .gitignore, so it won't exist on a fresh clone or CI checkout. fs.writeFile("e2e/.auth/seed-result.json", ...) does not create missing parent directories; it throws ENOENT: no such file or directory. This crashes the global-setup before any test runs.
Playwright's own storageState({ path }) call (line 38) handles directory creation internally, so that write is safe — but the manual writeFile here is not.
Add an mkdir before the write:
| const result = await seed(prisma); | |
| const fs = await import("fs/promises"); | |
| await fs.writeFile( | |
| "e2e/.auth/seed-result.json", | |
| JSON.stringify(result, null, 2), | |
| ); | |
| const fs = await import("fs/promises"); | |
| await fs.mkdir("e2e/.auth", { recursive: true }); | |
| await fs.writeFile( | |
| "e2e/.auth/seed-result.json", | |
| JSON.stringify(result, null, 2), | |
| ); |
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/global-setup.ts
Line: 14-20
Comment:
**`e2e/.auth/` directory is never created — CI will throw ENOENT**
`e2e/.auth/` is listed in `.gitignore`, so it won't exist on a fresh clone or CI checkout. `fs.writeFile("e2e/.auth/seed-result.json", ...)` does **not** create missing parent directories; it throws `ENOENT: no such file or directory`. This crashes the global-setup before any test runs.
Playwright's own `storageState({ path })` call (line 38) handles directory creation internally, so that write is safe — but the manual `writeFile` here is not.
Add an `mkdir` before the write:
```suggestion
const fs = await import("fs/promises");
await fs.mkdir("e2e/.auth", { recursive: true });
await fs.writeFile(
"e2e/.auth/seed-result.json",
JSON.stringify(result, null, 2),
);
```
How can I resolve this? If you propose a fix, please make it concise.|
|
||
| test("should show deployment badge on pipeline list after deploy", async ({ | ||
| page, | ||
| pipelinesPage, | ||
| sidebar, | ||
| }) => { | ||
| await sidebar.navigateTo("Pipelines"); | ||
| await page.waitForLoadState("networkidle"); | ||
|
|
||
| await pipelinesPage.expectPipelineInList("E2E Test Pipeline"); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
"should show deployment badge on pipeline list after deploy" only asserts that the pipeline name appears in the list (expectPipelineInList). It never calls pipelinesPage.expectDeploymentBadge("E2E Test Pipeline"), which is the method that actually checks the badge (verifying the row no longer says "Draft"). The test as written will pass even if the pipeline never deployed successfully — it's just a duplicate of the visibility check already covered by the pipeline-crud tests.
Call expectDeploymentBadge to make the assertion match the test's stated intent:
await pipelinesPage.expectPipelineInList("E2E Test Pipeline");
await pipelinesPage.expectDeploymentBadge("E2E Test Pipeline");Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/tests/deploy.spec.ts
Line: 26-37
Comment:
**Test name promises a deployment badge check that isn't performed**
`"should show deployment badge on pipeline list after deploy"` only asserts that the pipeline name appears in the list (`expectPipelineInList`). It never calls `pipelinesPage.expectDeploymentBadge("E2E Test Pipeline")`, which is the method that actually checks the badge (verifying the row no longer says "Draft"). The test as written will pass even if the pipeline never deployed successfully — it's just a duplicate of the visibility check already covered by the pipeline-crud tests.
Call `expectDeploymentBadge` to make the assertion match the test's stated intent:
```typescript
await pipelinesPage.expectPipelineInList("E2E Test Pipeline");
await pipelinesPage.expectDeploymentBadge("E2E Test Pipeline");
```
How can I resolve this? If you propose a fix, please make it concise.| await loginPage.goto(); | ||
| await loginPage.login("e2e@test.local", "TestPassword123!"); | ||
| await loginPage.expectRedirectedToDashboard(); |
There was a problem hiding this comment.
TEST_USER constants
"e2e@test.local" and "TestPassword123!" are repeated inline here (and again on lines 27, 40–41) rather than using the TEST_USER constants already defined in e2e/helpers/constants.ts. If the seed values ever change, these tests would silently diverge from the seed data.
| await loginPage.goto(); | |
| await loginPage.login("e2e@test.local", "TestPassword123!"); | |
| await loginPage.expectRedirectedToDashboard(); | |
| await loginPage.login(TEST_USER.email, TEST_USER.password); |
Import at the top of the file:
import { TEST_USER } from "../helpers/constants";This also applies to the other three credential literals in this file (lines 27, 40, 41).
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/tests/auth.spec.ts
Line: 14-16
Comment:
**Credentials hardcoded instead of using `TEST_USER` constants**
`"e2e@test.local"` and `"TestPassword123!"` are repeated inline here (and again on lines 27, 40–41) rather than using the `TEST_USER` constants already defined in `e2e/helpers/constants.ts`. If the seed values ever change, these tests would silently diverge from the seed data.
```suggestion
await loginPage.login(TEST_USER.email, TEST_USER.password);
```
Import at the top of the file:
```typescript
import { TEST_USER } from "../helpers/constants";
```
This also applies to the other three credential literals in this file (lines 27, 40, 41).
How can I resolve this? If you propose a fix, please make it concise.- Fix react-hooks/rules-of-hooks false positive in Playwright fixtures - Fix waitForURL glob matching login page (P1) - Add mkdir for e2e/.auth directory on fresh clone (P1) - Add expectDeploymentBadge call in deploy test (P2) - Use TEST_USER constants instead of hardcoded credentials (P2) - Remove unused imports (Locator, expect, page, toast)
* feat: install Playwright and configure E2E test infrastructure * feat: add E2E test helpers — constants, seed, cleanup * feat: add E2E page objects — login, pipelines, editor, fleet, alerts, shared components * feat: add Playwright fixtures, global setup with auth, and teardown * feat: add 5 E2E test specs — auth, pipeline CRUD, deploy, fleet, alerts * feat: add E2E CI workflow and Docker Compose for local Postgres * chore: add wait-on for E2E CI server readiness check * fix: resolve E2E setup issues found during verification - Add e2e/tsconfig.json with moduleResolution: node so Playwright can resolve the generated Prisma client and path aliases at runtime - Fix cleanup.ts: prisma.alertDelivery → prisma.deliveryAttempt to match the actual Prisma schema (model is DeliveryAttempt, not AlertDelivery) - src/generated/prisma must be generated via `npx prisma generate` before running E2E tests (added to CI workflow already) * chore: run E2E tests only on release tags and manual dispatch * fix: resolve CI lint errors and Greptile P1/P2 findings - Fix react-hooks/rules-of-hooks false positive in Playwright fixtures - Fix waitForURL glob matching login page (P1) - Add mkdir for e2e/.auth directory on fresh clone (P1) - Add expectDeploymentBadge call in deploy test (P2) - Use TEST_USER constants instead of hardcoded credentials (P2) - Remove unused imports (Locator, expect, page, toast) * fix: restore page param in pipeline delete test
1 participant
Summary
e2e.yml) with Postgres service container, nightly cron + PR triggers, artifact upload on failureFiles
playwright.config.ts— setup project with storageState auth, chromium-only, global teardowne2e/helpers/— constants, seed script, cleanup scripte2e/pages/— 5 page objects + 3 shared components (sidebar, toast, deploy dialog)e2e/fixtures/test.fixture.ts— extended test with all page object fixturese2e/global-setup.ts— DB seed + UI login + storageState savee2e/tests/— auth, pipeline-crud, deploy, fleet, alerts specs.github/workflows/e2e.yml— CI with Postgres service, wait-on, artifact uploade2e/docker-compose.e2e.yml— local Postgres for developmentTest plan
npx playwright test --listshows 15 tests across 6 filesnpx tsc --project e2e/tsconfig.json --noEmitcompiles cleanlydocker compose -f e2e/docker-compose.e2e.yml up -d+pnpm dev+pnpm test:e2elocally