Skip to content

Adding playwright config for running console e2e. #1706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sriroopar wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Adding playwright config for running console e2e. #1706

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions .tekton/integration-tests/pipelines/lightspeed-console-e2e-test-pipeline-pf5.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
name: ols-console-pre-commits
spec:
description: |
Runs OpenShift Lightspeed console e2e (Cypress) on a Konflux ephemeral OpenShift cluster (EaaS).
Runs OpenShift Lightspeed console e2e (Playwright) on a Konflux ephemeral OpenShift cluster (EaaS).
Provisions the cluster, installs the operator bundle from the snapshot, runs console tests,
collects artifacts, then deprovisions.
params:
Expand Down Expand Up @@ -172,24 +172,25 @@ spec:
- name: credentials
mountPath: /credentials
env:
- name: CYPRESS_KUBECONFIG_PATH
- name: KUBECONFIG_PATH
value: "/credentials/$(steps.get-kubeconfig.results.kubeconfig)"
- name: CYPRESS_LOGIN_IDP
- name: LOGIN_IDP
value: "kube:admin"
- name: LLM_TOKEN_PATH
value: "/var/run/openai/token"
- name: COMMIT_SHA
value: "$(params.commit)"
- name: PASSWORD_PATH
value: "/credentials/$(steps.get-kubeconfig.results.passwordPath)"
- name: CYPRESS_BASE_URL
- name: BASE_URL
value: "$(steps.get-kubeconfig.results.consoleURL)"
- name: BUNDLE_COMMIT_SHA
value: "$(params.commit)"
- name: CYPRESS_BUNDLE_IMAGE
- name: BUNDLE_IMAGE
value: "$(params.bundle-image)"
image: cypress/browsers:26.0.0
image: mcr.microsoft.com/playwright:v1.60.0-noble

@coderabbitai coderabbitai Bot Jun 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Verify Playwright Docker image exists and check for vulnerabilities

# Check if the image exists and can be pulled
echo "Checking if image exists..."
docker manifest inspect mcr.microsoft.com/playwright:v1.60.0-noble >/dev/null 2>&1 && echo "✓ Image exists" || echo "✗ Image not found"

# Check available Playwright versions
echo -e "\nFetching recent Playwright releases..."
gh api repos/microsoft/playwright/releases --paginate | jq -r '.[].tag_name' | head -20

Repository: openshift/lightspeed-operator

Length of output: 319

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Checking likely existing Playwright image tags..."
for tag in "v1.60.0" "v1.60.0-noble" "v1.60.0-jammy" "v1.60.0-focal" "v1.60.0-bookworm" "v1.60.0-oraclelinux9"; do
  printf " - %s: " "$tag"
  if docker manifest inspect "mcr.microsoft.com/playwright:${tag}" >/dev/null 2>&1; then
    echo "✓ exists"
  else
    echo "✗ not found"
  fi
done

Repository: openshift/lightspeed-operator

Length of output: 307

🌐 Web query:

mcr.microsoft.com/playwright v1.60.0 noble tag exists

💡 Result:

Yes, the Docker tag mcr.microsoft.com/playwright:v1.60.0-noble exists in the Microsoft Artifact Registry [1]. Official Playwright documentation confirms that they publish images for v1.60.0 based on Ubuntu 24.04 LTS (Noble Numbat) using this tag [2][3].

Citations:

🏁 Script executed:

#!/bin/bash
set -euo pipefail

IMAGE="mcr.microsoft.com/playwright:v1.60.0-noble"

echo "Checking published tag page..."
curl -fsSL "https://mcr.microsoft.com/en-us/artifact/mar/playwright/tag/v1.60.0-noble" >/dev/null \
  && echo "✓ tag page reachable" || echo "✗ tag page not reachable"

echo "Scanning image (requires trivy)..."
if command -v trivy >/dev/null 2>&1; then
  docker pull "$IMAGE"
  trivy image --severity HIGH,CRITICAL --no-progress "$IMAGE"
else
  echo "trivy not installed; run trivy/grype in CI against $IMAGE and fail on HIGH/CRITICAL"
fi

Repository: openshift/lightspeed-operator

Length of output: 226

🏁 Script executed:

#!/bin/bash
set -euo pipefail

IMAGE="mcr.microsoft.com/playwright:v1.60.0-noble"

echo "Checking published tag..."
curl -fsSL "https://mcr.microsoft.com/en-us/artifact/mar/playwright/tag/v1.60.0-noble" >/dev/null \
  && echo "✓ Tag page reachable" || echo "✗ Tag page not reachable"

if command -v trivy >/dev/null 2>&1 && command -v docker >/dev/null 2>&1; then
  docker pull "$IMAGE"
  trivy image --severity HIGH,CRITICAL --no-progress "$IMAGE"
else
  echo "Run this check in CI with docker + trivy installed to scan $IMAGE (fail on HIGH/CRITICAL)."
fi

Repository: openshift/lightspeed-operator

Length of output: 247

🏁 Script executed:

#!/bin/bash
set -euo pipefail

IMAGE="mcr.microsoft.com/playwright:v1.60.0-noble"

command -v docker >/dev/null 2>&1 || { echo "docker not found"; exit 1; }
command -v trivy >/dev/null 2>&1 || { echo "trivy not found"; exit 1; }

docker pull "$IMAGE"
trivy image --severity HIGH,CRITICAL --no-progress "$IMAGE"

# Optional (recommended): pin by digest for reproducibility
# DIGEST="$(docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE" | cut -d@ -f2)"
# echo "Pinned digest: $DIGEST"

Repository: openshift/lightspeed-operator

Length of output: 90

🏁 Script executed:

#!/bin/bash
set -euo pipefail

IMAGE="mcr.microsoft.com/playwright:v1.60.0-noble"

# In CI where docker + trivy are available
command -v docker >/dev/null 2>&1 || { echo "docker not found"; exit 1; }
command -v trivy >/dev/null 2>&1 || { echo "trivy not found"; exit 1; }

docker pull "$IMAGE"
trivy image --severity HIGH,CRITICAL --no-progress "$IMAGE"

# Optional: pin to immutable digest (recommended)
# DIGEST="$(docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE" | awk -F@ '{print $2}')"
# echo "$IMAGE@$DIGEST"

Repository: openshift/lightspeed-operator

Length of output: 90

🏁 Script executed:

#!/bin/bash
set -euo pipefail

IMAGE="mcr.microsoft.com/playwright:v1.60.0-noble"

command -v docker >/dev/null 2>&1 || { echo "docker not found (run in CI)"; exit 1; }
command -v trivy >/dev/null 2>&1 || { echo "trivy not found (install in CI)"; exit 1; }

docker pull "$IMAGE"
trivy image --severity HIGH,CRITICAL --no-progress "$IMAGE"

# Optional (recommended): pin to digest
# DIGEST="$(docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE" | awk -F@ '{print $2}')"
# echo "$IMAGE@$DIGEST"

Repository: openshift/lightspeed-operator

Length of output: 102

🏁 Script executed:

#!/bin/bash
set -euo pipefail

IMAGE="mcr.microsoft.com/playwright:v1.60.0-noble"

command -v docker >/dev/null 2>&1 || { echo "docker not found (run in CI)"; exit 1; }
command -v trivy >/dev/null 2>&1 || { echo "trivy not found (install in CI)"; exit 1; }

docker pull "$IMAGE"
trivy image --severity HIGH,CRITICAL --no-progress "$IMAGE"

# Optional (recommended): pin to immutable digest
# DIGEST="$(docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE" | awk -F@ '{print $2}')"
# echo "Pinned digest: $IMAGE@$DIGEST"

Repository: openshift/lightspeed-operator

Length of output: 102

🏁 Script executed:

#!/bin/bash
set -euo pipefail

IMAGE="mcr.microsoft.com/playwright:v1.60.0-noble"

command -v docker >/dev/null 2>&1 || { echo "docker not found"; exit 1; }
command -v trivy >/dev/null 2>&1 || { echo "trivy not found"; exit 1; }

docker pull "$IMAGE"
trivy image --severity HIGH,CRITICAL --no-progress "$IMAGE"

# Optional: pin by digest for reproducibility
# DIGEST="$(docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE" | awk -F@ '{print $2}')"
# echo "$IMAGE@$DIGEST"

Repository: openshift/lightspeed-operator

Length of output: 90

🏁 Script executed:

#!/bin/bash
set -euo pipefail

IMAGE="mcr.microsoft.com/playwright:v1.60.0-noble"

command -v trivy >/dev/null 2>&1 || { echo "trivy not found (install in CI)"; exit 1; }

# Scans the remote image directly (no docker daemon required)
trivy image --severity HIGH,CRITICAL --no-progress "$IMAGE"

# Optional: pin to immutable digest (do in CI using your preferred tooling, e.g., crane/skopeo)
# DIGEST="$(crane digest "$IMAGE")"
# echo "Pinned image: $IMAGE@$DIGEST"

Repository: openshift/lightspeed-operator

Length of output: 45074

Address HIGH vulnerabilities in Playwright image used by the pipeline

In .tekton/integration-tests/pipelines/lightspeed-console-e2e-test-pipeline-pf5.yaml (line 191), mcr.microsoft.com/playwright:v1.60.0-noble is an official Microsoft Playwright Ubuntu 24.04 “Noble” image, but Trivy reports 4 HIGH vulnerabilities (0 CRITICAL): CVE-2025-3887 (gstreamer1.0-plugins-bad), CVE-2026-45447 (libssl3t64), and CVE-2026-33671 (picomatch). Update to a newer patched Playwright image tag/digest (or justify risk acceptance if this is intentionally tolerated for e2e-only usage).

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
@.tekton/integration-tests/pipelines/lightspeed-console-e2e-test-pipeline-pf5.yaml
at line 191, The pipeline is using a vulnerable Playwright image
"mcr.microsoft.com/playwright:v1.60.0-noble" which Trivy flagged for multiple
HIGH CVEs; replace that image reference with a newer patched Playwright image
tag or digest (e.g., a more recent v1.x-noble tag or an explicit digest) to
eliminate CVE-2025-3887, CVE-2026-45447 and CVE-2026-33671, or if you intend to
accept the risk, add an explicit justification comment in the pipeline and a
documented risk acceptance ticket; update the image string in the pipeline task
spec where "mcr.microsoft.com/playwright:v1.60.0-noble" appears (or pin to an
immutable digest) and re-run vulnerability scans to verify remediation.

script: |
#!/bin/bash
set -euo pipefail
apt update && apt install -y jq git curl
cd /home
Expand All @@ -198,7 +199,7 @@ spec:
git remote add origin https://github.com/openshift/lightspeed-operator.git
git fetch --depth=1 --filter=blob:none origin "${COMMIT_SHA}"
git checkout "${COMMIT_SHA}"
git show "${COMMIT_SHA}:.tekton/integration-tests/scripts/run-console-cypress-tests.sh" | bash -s -- "lightspeed-console-plugin-pf5" "latest-4.18"
git show "${COMMIT_SHA}:.tekton/integration-tests/scripts/run-console-playwright-tests.sh" | bash -s -- "lightspeed-console-plugin-pf5" "latest-4.18"
- name: gather-cluster-resources
onError: continue
ref:
Expand Down
15 changes: 8 additions & 7 deletions .tekton/integration-tests/pipelines/lightspeed-console-e2e-test-pipeline-pf6.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
name: ols-console-pre-commits
spec:
description: |
Runs OpenShift Lightspeed console e2e (Cypress) on a Konflux ephemeral OpenShift cluster (EaaS).
Runs OpenShift Lightspeed console e2e (Playwright) on a Konflux ephemeral OpenShift cluster (EaaS).
Provisions the cluster, installs the operator bundle from the snapshot, runs console tests,
collects artifacts, then deprovisions.
params:
Expand Down Expand Up @@ -172,24 +172,25 @@ spec:
- name: credentials
mountPath: /credentials
env:
- name: CYPRESS_KUBECONFIG_PATH
- name: KUBECONFIG_PATH
value: "/credentials/$(steps.get-kubeconfig.results.kubeconfig)"
- name: CYPRESS_LOGIN_IDP
- name: LOGIN_IDP
value: "kube:admin"
- name: LLM_TOKEN_PATH
value: "/var/run/openai/token"
- name: COMMIT_SHA
value: "$(params.commit)"
- name: PASSWORD_PATH
value: "/credentials/$(steps.get-kubeconfig.results.passwordPath)"
- name: CYPRESS_BASE_URL
- name: BASE_URL
value: "$(steps.get-kubeconfig.results.consoleURL)"
- name: BUNDLE_COMMIT_SHA
value: "$(params.commit)"
- name: CYPRESS_BUNDLE_IMAGE
- name: BUNDLE_IMAGE
value: "$(params.bundle-image)"
image: cypress/browsers:26.0.0
image: mcr.microsoft.com/playwright:v1.60.0-noble
script: |
#!/bin/bash
set -euo pipefail
apt update && apt install -y jq git curl
cd /home
Expand All @@ -198,7 +199,7 @@ spec:
git remote add origin https://github.com/openshift/lightspeed-operator.git
git fetch --depth=1 --filter=blob:none origin "${COMMIT_SHA}"
git checkout "${COMMIT_SHA}"
git show "${COMMIT_SHA}:.tekton/integration-tests/scripts/run-console-cypress-tests.sh" | bash -s -- "lightspeed-console-plugin" "latest-4.19"
git show "${COMMIT_SHA}:.tekton/integration-tests/scripts/run-console-playwright-tests.sh" | bash -s -- "lightspeed-console-plugin" "latest-4.19"
- name: gather-cluster-resources
onError: continue
ref:
Expand Down
47 changes: 26 additions & 21 deletions ...ests/scripts/run-console-cypress-tests.sh → ...s/scripts/run-console-playwright-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,28 +1,33 @@
#!/usr/bin/env bash
# Konflux console Cypress: run from lightspeed-operator repo root at ${COMMIT_SHA}
# Konflux console Playwright: run from lightspeed-operator repo root at ${COMMIT_SHA}
# (Tekton performs apt + git init/fetch/checkout before piping this script).
#
# Usage (from Tekton, cwd = /home/lightspeed-operator):
# git show "${COMMIT_SHA}:.tekton/integration-tests/scripts/run-console-cypress-tests.sh" \
# git show "${COMMIT_SHA}:.tekton/integration-tests/scripts/run-console-playwright-tests.sh" \
# | bash -s -- "<related_images.json component name>" "<oc client channel e.g. latest-4.18>"
#
# Args:
# $1 Name in related_images.json for the console plugin (e.g. lightspeed-console-plugin-pf5 or lightspeed-console-plugin)
# $2 OpenShift client channel for install-oc-if-missing.sh (e.g. latest-4.18), aligned with the ephemeral cluster minor
#
# Env: COMMIT_SHA, CYPRESS_BASE_URL, CYPRESS_CONSOLE_IMAGE, CYPRESS_KUBECONFIG_PATH, PASSWORD_PATH, etc.
# Env: COMMIT_SHA, BASE_URL, KUBECONFIG_PATH, PASSWORD_PATH, LOGIN_IDP, BUNDLE_IMAGE, etc.

set -euo pipefail

console_component="${1:?usage: $0 <related_images component name> <ocp channel e.g. latest-4.18>}"
ocp_channel="${2:?usage: $0 <related_images name> <ocp channel>}"

echo "COMMIT_SHA: ${COMMIT_SHA}"
echo "CYPRESS_BASE_URL: ${CYPRESS_BASE_URL:-}"
echo "CYPRESS_CONSOLE_IMAGE: ${CYPRESS_CONSOLE_IMAGE:-}"
echo "BASE_URL: ${BASE_URL:-}"
echo "CONSOLE_IMAGE: ${CONSOLE_IMAGE:-}"
echo "---------------------------------------------"
export CYPRESS_LOGIN_PASSWORD="$(cat "${PASSWORD_PATH}")"
echo "(CYPRESS_LOGIN_PASSWORD set from PASSWORD_PATH; not echoed)"
if [[ ! -r "${PASSWORD_PATH}" ]]; then
echo "ERROR: PASSWORD_PATH '${PASSWORD_PATH}' is not readable" >&2
exit 1
fi
LOGIN_PASSWORD="$(cat "${PASSWORD_PATH}")"
export LOGIN_PASSWORD
echo "(LOGIN_PASSWORD set from PASSWORD_PATH; not echoed)"
echo "---------------------------------------------"

git show "${COMMIT_SHA}:.tekton/integration-tests/scripts/install-oc-if-missing.sh" | bash -s -- "${ocp_channel}"
Expand All @@ -41,7 +46,7 @@ echo "---------------------------------------------"
operator-sdk version
echo "---------------------------------------------"

# Valid XDG path for Cypress/Electron; must not reuse $PATH (breaks browser runtime).
# Valid XDG path for Playwright/Chromium; must not reuse $PATH (breaks browser runtime).
XDG_RUNTIME_DIR="${HOME:-/root}/.cache/xdgr"
mkdir -p "${XDG_RUNTIME_DIR}"
export XDG_RUNTIME_DIR
Expand All @@ -64,32 +69,32 @@ echo "---------------------------------------------"
NODE_OPTIONS=--max-old-space-size=4096 npm ci --omit=optional --no-fund
npx cypress install
echo "---------------------------------------------"
export CYPRESS_LOGIN_PASSWORD="$(cat "${PASSWORD_PATH}")"
# Ephemeral clusters + console OAuth + plugin proxy are slow; before() often runs bundle then UI.
export CYPRESS_defaultCommandTimeout="${CYPRESS_defaultCommandTimeout:-120000}"
export CYPRESS_requestTimeout="${CYPRESS_requestTimeout:-120000}"
export CYPRESS_pageLoadTimeout="${CYPRESS_pageLoadTimeout:-180000}"
export CYPRESS_responseTimeout="${CYPRESS_responseTimeout:-180000}"
export CYPRESS_execTimeout="${CYPRESS_execTimeout:-600000}"

run_cypress() {
NO_COLOR=1 npx cypress run "$@"
# Install Playwright browsers (chromium only, with OS deps).
npx playwright install --with-deps chromium
echo "---------------------------------------------"

# Enable Playwright CI mode (forbidOnly, etc.).
export CI=true

run_playwright() {
npx playwright test "$@"
}

set +e
run_cypress
run_playwright
err_status=$?
if [[ "${err_status}" -ne 0 ]]; then
echo "---------------------------------------------"
echo "Cypress exited ${err_status}; waiting 30s for console/plugin then retrying once..."
echo "Playwright exited ${err_status}; waiting 30s for console/plugin then retrying once..."
sleep 30
run_cypress
run_playwright
err_status=$?
fi
echo -n "${err_status}" >/workspace/cypress-exit-code
echo "---------------------------------------------"
ls ./gui_test_screenshots
mv ./gui_test_screenshots /workspace/artifacts/
set -e
echo "Cypress exit code: ${err_status}"
echo "Playwright exit code: ${err_status}"
exit "${err_status}"