Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ echo "127.0.0.1 iam.zenko.local s3-local-file.zenko.local keycloak.zenko.local s
bash .github/scripts/end2end/run-e2e-ctst.sh @yourTag
```

See [tests/ctst/README.md](../tests/ctst/README.md) for more details.
See [tests/functional/ctst/README.md](../tests/functional/ctst/README.md) for more details.

## Running e2e tests in the codespace

Expand Down
6 changes: 3 additions & 3 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -96,11 +96,11 @@
"vscode": {
"settings": {
"cucumber.features": [
"tests/ctst/features/**/*.feature"
"tests/functional/ctst/features/**/*.feature"
],
"cucumber.glue": [
"tests/ctst/common/*.ts",
"tests/ctst/steps/**/*.ts"
"tests/functional/ctst/common/*.ts",
"tests/functional/ctst/steps/**/*.ts"
]
},
"extensions": [
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ done


# Extract sorbet & drctl binaries for out-of-cluster test execution
CTST_DIR="./tests/ctst"
CTST_DIR="./tests/functional/ctst"
SORBET_IMAGE=$(yq eval '.sorbet | .sourceRegistry + "/" + .image' solution/deps.yaml)
SORBET_TAG=$(yq eval '.sorbet.tag' solution/deps.yaml)
DRCTL_IMAGE=$(yq eval '.drctl | .sourceRegistry + "/" + .image' solution/deps.yaml)
Expand Down
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ Fixes ZENKO-XXXX.

<!--
When adding new behavior tests, please follow the
`tests/ctst/HOW_TO_WRITE_TESTS.md` guidelines.
`tests/functional/ctst/HOW_TO_WRITE_TESTS.md` guidelines.
-->
54 changes: 54 additions & 0 deletions .github/actions/setup-node-env/action.yaml
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

end2end.yaml file was like 700 lines, with like ~6 nodes setup so I think this is better

Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
name: Setup Node environment
description: >
Create a GitHub App token for private repo access, configure git,
set up Node.js with yarn cache, and install test dependencies.

inputs:
app-id:
description: GitHub App ID
required: true
private-key:
description: GitHub App private key
required: true
repositories:
description: Newline-separated list of private repositories to grant access to
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before the refactor, each job explicitly requested only the repos it consumes:
lint-ctst → cli-testing
end2end-2-shards-http → zenko-operator
end2end-sharded → metadata, zenko-operator
now, unless explicitly overridden (and no caller overrides), every job receives a token with access to all 3 repos. And check-workflows ends up with that token while it previously needed no private repo at all. I don't know if that is okay but rather point it as it's hidden inside this composite action.
I think we should remove the default, force each caller to pass the exact list (like the original code). If you want, keep a minimal default of cli-testing and we can document the convention clearly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I noticed this stuff, not sure it's super important, but yeah I removed the default value and re-add explicit repo list

required: true

outputs:
token:
description: The generated GitHub App token
value: ${{ steps.app-token.outputs.token }}

runs:
using: composite
steps:
- name: Get token to access private repositories
uses: actions/create-github-app-token@v3
id: app-token
with:
app-id: ${{ inputs.app-id }}
private-key: ${{ inputs.private-key }}
owner: ${{ github.repository_owner }}
repositories: ${{ inputs.repositories }}
- name: Configure GIT
shell: bash
run: |
git config --global url."https://x-access-token:${GIT_ACCESS_TOKEN}@github.com/".insteadOf "https://github.com/"
git config --global --add url."https://x-access-token:${GIT_ACCESS_TOKEN}@github.com/".insteadOf "git@github.com:"
git config --global --add url."https://x-access-token:${GIT_ACCESS_TOKEN}@github.com/".insteadOf "ssh://git@github.com/"
env:
GIT_ACCESS_TOKEN: ${{ steps.app-token.outputs.token }}
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: '24'
cache: yarn
cache-dependency-path: tests/functional/yarn.lock
- name: Install test dependencies
shell: bash
working-directory: tests/functional
run: >-
yarn install --frozen-lockfile --network-concurrency 1 || (
yarn cache clean && yarn install --frozen-lockfile --network-concurrency 1
)
4 changes: 2 additions & 2 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Key paths:
- `solution/zenkoversion.yaml` — `ZenkoVersion` CR template (dashboards, policies, feature flags, capabilities, location types)
- `solution/kafka/Dockerfile`, `solution/kafka-connect/Dockerfile` — Scality Kafka + Kafka Connect images
- `solution-base/mongodb/` — MongoDB Helm charts and patches
- `tests/ctst/` — TypeScript Cucumber end-to-end tests
- `tests/functional/ctst/` — TypeScript Cucumber end-to-end tests
- `tests/workflows/` — TypeScript Jest tests for CI tooling
- `tests/zenko_tests/` — Python + Node.js integration tests
- `monitoring/` — Prometheus rules, Grafana dashboards
Expand All @@ -27,7 +27,7 @@ When reviewing a PR, analyze changes against the following. Post inline comments
| **Dockerfiles** (`solution/kafka/`, `solution/kafka-connect/`...) | Base images pinned by tag or digest, no secrets baked in, no unnecessary `COPY . .`, reasonable layer count, user is non-root where possible. |
| **Helm charts & K8s manifests** (`solution-base/mongodb/charts/`, `monitoring/`) | Resource requests/limits set, label selectors match, no hard-coded namespaces, `securityContext` present, ServiceAccount scoping minimal. Any breaking chart-value renames documented in an upgrade note. |
| **Chart upgrade path** (`solution-base/mongodb/patches/`, `how_to_upgrade.md`) | If chart version or MongoDB version changes, upgrade notes are updated and patches still apply cleanly. |
| **TypeScript tests** (`tests/ctst/`, `tests/workflows/`) | Proper `async`/`await`, no swallowed promise rejections, Cucumber step definitions register correctly, no accidental `.only` / `.skip`, correct use of World context in ctst. |
| **TypeScript tests** (`tests/functional/ctst/`, `tests/workflows/`) | Proper `async`/`await`, no swallowed promise rejections, Cucumber step definitions register correctly, no accidental `.only` / `.skip`, correct use of World context in ctst. |
| **Python tests** (`tests/zenko_tests/`) | No bare `except:`, specific exception types, consistency with existing style, `requirements.txt` kept in sync. |
| **CI workflows** (`.github/workflows/`) | Actions pinned (tag or SHA), secrets not echoed to logs, `permissions:` block scoped minimally, reusable-workflow inputs/secrets wired correctly, runner labels valid for Scality infra. |
| **Monitoring changes** (`monitoring/`) | PromQL expressions valid, alert labels/annotations follow existing conventions, dashboard panels reference real metrics, Grafana JSON not corrupted by export tooling. |
Expand Down
6 changes: 3 additions & 3 deletions .github/scripts/end2end/configure-e2e.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ kubectl run kafka-topics \
kafka-topics.sh --create --topic $NOTIF_ALT_DEST_TOPIC --bootstrap-server $KAFKA_HOST_PORT --if-not-exists"

# Run configuration.py directly
ZENKO_TESTS_DIR="$DIR/../../../tests/zenko_tests"
pip3 install --break-system-packages -r "$ZENKO_TESTS_DIR/requirements.txt"
CONFIGURATION_TESTS_DIR="$DIR/../../../tests/functional/configuration_setup"
pip3 install --break-system-packages -r "$CONFIGURATION_TESTS_DIR/requirements.txt"

cd "$ZENKO_TESTS_DIR"
cd "$CONFIGURATION_TESTS_DIR"

envsubst < e2e-config.yaml.template > e2e-config.yaml

Expand Down
12 changes: 6 additions & 6 deletions .github/scripts/end2end/run-e2e-ctst.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,18 @@ DIR=$(cd "$(dirname "$0")" && pwd)

source "$DIR/setup-e2e-env.sh"

CTST_DIR="$(cd "$DIR/../../../tests/ctst" && pwd)"
cd "$CTST_DIR"
mkdir -p reports
FUNCTIONAL_TESTS_DIR="$(cd "$DIR/../../../tests/functional" && pwd)"
cd "$FUNCTIONAL_TESTS_DIR"
mkdir -p ctst/reports

export SDK=true # Cli-testing also has a cli mode, not really used in practice
yarn cucumber-js \
--config cucumber.config.cjs \
--config ctst/cucumber.config.cjs \
--tags "${TAGS}" \
--world-parameters "${CTST_WORLD_PARAMETERS}" \
--parallel "${PARALLEL_RUNS}" \
--retry 3 \
--retry-tag-filter @Flaky \
--format pretty \
--format html:reports/report.html \
--format junit:reports/report.xml
--format html:ctst/reports/report.html \
--format junit:ctst/reports/report.xml
7 changes: 1 addition & 6 deletions .github/scripts/end2end/setup-e2e-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ else
--group=system:serviceaccounts 2>/dev/null || true

# --- 15b. Install sorbet & drctl binaries for CTST ---
CTST_DIR="$(cd "${ZENKO_ROOT}/tests/ctst" && pwd)"
CTST_DIR="$(cd "${ZENKO_ROOT}/tests/functional/ctst" && pwd)"
SORBET_IMAGE=$(yq eval '.sorbet | .sourceRegistry + "/" + .image' "${ZENKO_ROOT}/solution/deps.yaml")
SORBET_TAG=$(yq eval '.sorbet.tag' "${ZENKO_ROOT}/solution/deps.yaml")
DRCTL_IMAGE=$(yq eval '.drctl | .sourceRegistry + "/" + .image' "${ZENKO_ROOT}/solution/deps.yaml")
Expand Down Expand Up @@ -345,11 +345,6 @@ EOF

fi # SKIP_CTST

# --- 17. Install node dependencies ---
NODE_TESTS_DIR="$(cd "${ZENKO_ROOT}/tests/zenko_tests/node_tests" && pwd)"
cd "$NODE_TESTS_DIR"
yarn install --frozen-lockfile

# --- 18. Persist exports for subsequent CI steps ---
if [ -n "${GITHUB_ENV:-}" ]; then # Don't do it for Codespace
echo "TOKEN=$TOKEN" >> "$GITHUB_ENV"
Expand Down
Loading
Loading