diff --git a/docs/workflows/bin-package-18.04.md b/docs/workflows/bin-package-18.04.md new file mode 100644 index 000000000..ffc725cdc --- /dev/null +++ b/docs/workflows/bin-package-18.04.md @@ -0,0 +1,29 @@ +# bin-package-18.04 + +**File:** `.github/workflows/bin-package-18.04.yaml` +**Trigger:** Called by other workflows (`workflow_call`) + +## Purpose + +Builds a single runtime package using an **Ubuntu 18.04** container. This is identical to `bin-package` but forces the older Ubuntu version for packages that require it (e.g. `mdadm`). + +## Inputs + +| Input | Required | Description | +|-------|----------|-------------| +| `package` | Yes | Name of the package to build | + +## Secrets + +| Secret | Description | +|--------|-------------| +| `token` | Hub JWT token for publishing flists | + +## Steps + +1. **Checkout** the repository +2. **Set tag**: Determines the version reference — uses git tag name for tag pushes, or short SHA for branch pushes +3. **Setup basesystem**: Runs `bins/bins-extra.sh --package basesystem` to install build prerequisites +4. **Build package**: Runs `bins/bins-extra.sh --package ` to compile the binary +5. **Publish flist**: Uploads the built binary as an flist to `tf-autobuilder/.flist` on the hub +6. **Tagging**: On `main`, `testdeploy`, or version tags (`v*`), creates a tag link `/.flist` pointing to the published flist diff --git a/docs/workflows/bin-package-no-tag.md b/docs/workflows/bin-package-no-tag.md new file mode 100644 index 000000000..caab54940 --- /dev/null +++ b/docs/workflows/bin-package-no-tag.md @@ -0,0 +1,29 @@ +# bin-package-no-tag + +**File:** `.github/workflows/bin-package-no-tag.yaml` +**Trigger:** Called by other workflows (`workflow_call`) + +## Purpose + +Builds a single runtime package using Ubuntu 20.04 but **never tags** the built binary. This means any package built with this workflow never becomes part of a ZOS installation release. Used for packages like `qsfs` and `traefik` that are published independently. + +## Inputs + +| Input | Required | Description | +|-------|----------|-------------| +| `package` | Yes | Name of the package to build | + +## Secrets + +| Secret | Description | +|--------|-------------| +| `token` | Hub JWT token for publishing flists | + +## Steps + +1. **Checkout** the repository +2. **Setup basesystem**: Installs build prerequisites via `bins/bins-extra.sh --package basesystem` +3. **Build package**: Compiles the binary via `bins/bins-extra.sh --package ` +4. **Publish flist**: Uploads the built binary as an flist to `tf-autobuilder/.flist` on the hub + +No tagging step is performed — the flist is published but not linked to any release version. diff --git a/docs/workflows/bin-package.md b/docs/workflows/bin-package.md new file mode 100644 index 000000000..49ce599ac --- /dev/null +++ b/docs/workflows/bin-package.md @@ -0,0 +1,29 @@ +# bin-package + +**File:** `.github/workflows/bin-package.yaml` +**Trigger:** Called by other workflows (`workflow_call`) + +## Purpose + +Builds a single runtime package using Ubuntu 20.04. The built binary is always published to `tf-autobuilder` and then tagged with the release version or the short SHA of the commit head (on main branch). This is the standard build workflow invoked by `bins.yaml` for most packages. + +## Inputs + +| Input | Required | Description | +|-------|----------|-------------| +| `package` | Yes | Name of the package to build | + +## Secrets + +| Secret | Description | +|--------|-------------| +| `token` | Hub JWT token for publishing flists | + +## Steps + +1. **Checkout** the repository +2. **Set tag**: Determines the version reference — uses git tag name for tag pushes (e.g. `v1.2.3`), or short SHA for branch pushes +3. **Setup basesystem**: Runs `apt update` then `bins/bins-extra.sh --package basesystem` +4. **Build package**: Compiles the binary via `bins/bins-extra.sh --package ` +5. **Publish flist**: On `main`, `testdeploy`, or version tags, uploads to `tf-autobuilder/.flist` +6. **Tagging**: On `main`, `testdeploy`, or version tags, creates a tag link `/.flist` pointing to the published flist diff --git a/docs/workflows/bins.md b/docs/workflows/bins.md new file mode 100644 index 000000000..0883966eb --- /dev/null +++ b/docs/workflows/bins.md @@ -0,0 +1,51 @@ +# bins (Runtime Packages) + +**File:** `.github/workflows/bins.yaml` +**Trigger:** Push to any branch, or version tags (`v*`) + +## Purpose + +Orchestrates the build of **all runtime packages** that ZOS depends on. Each package is built in parallel by calling one of the reusable workflows: + +- `bin-package.yaml` — standard build + tag (most packages) +- `bin-package-18.04.yaml` — build on Ubuntu 18.04 (for packages requiring older glibc) +- `bin-package-no-tag.yaml` — build without tagging (packages managed independently) + +## Packages + +### Standard packages (bin-package) + +| Package | Description | +|---------|-------------| +| `containerd` | Container runtime | +| `runc` | OCI runtime | +| `virtwhat` | Hypervisor detection | +| `yggdrasil` | Overlay network daemon | +| `rfs` | Remote filesystem | +| `hdparm` | Disk parameters tool | +| `corex` | Container interactive shell | +| `shimlogs` | Container log shim | +| `cloudhypervisor` | VM hypervisor | +| `tailstream` | Log tail streaming | +| `virtiofsd` | Virtio filesystem daemon | +| `vector` | Log/metrics collection | +| `nnc` | Network connectivity checker | +| `lshw` | Hardware lister | +| `cloudconsole` | Cloud console | +| `misc` | Miscellaneous tools | +| `iperf` | Network performance | +| `cpubench` | CPU benchmarking | +| `mycelium` | Mycelium network daemon | + +### Ubuntu 18.04 packages (bin-package-18.04) + +| Package | Description | +|---------|-------------| +| `mdadm` | RAID management | + +### No-tag packages (bin-package-no-tag) + +| Package | Description | +|---------|-------------| +| `qsfs` | Quantum Safe File System | +| `traefik` | Reverse proxy | diff --git a/docs/workflows/codeql-analysis.md b/docs/workflows/codeql-analysis.md new file mode 100644 index 000000000..ea4decae7 --- /dev/null +++ b/docs/workflows/codeql-analysis.md @@ -0,0 +1,18 @@ +# CodeQL Analysis + +**File:** `.github/workflows/codeql-analysis.yml` +**Trigger:** +- Push to `main` +- Pull requests targeting `main` +- Scheduled: every Wednesday at 11:00 UTC + +## Purpose + +Runs GitHub's CodeQL static analysis tool on the Go codebase to find security vulnerabilities and code quality issues. + +## Steps + +1. **Checkout** the repository (with depth 2 for PR head detection) +2. **Initialize CodeQL** for Go language scanning +3. **Autobuild** — CodeQL automatically detects and builds the Go project +4. **Perform analysis** — runs the CodeQL queries and reports findings diff --git a/docs/workflows/grid-deploy.md b/docs/workflows/grid-deploy.md new file mode 100644 index 000000000..2a8be731e --- /dev/null +++ b/docs/workflows/grid-deploy.md @@ -0,0 +1,21 @@ +# Grid Deploy + +**File:** `.github/workflows/grid-deploy.yaml` +**Trigger:** Manual dispatch (`workflow_dispatch`) + +## Purpose + +Deploys a specific ZOS version to a target grid environment (qa, testing, or production) by creating a cross-link on the hub. This effectively makes the specified version the "latest" for that grid. + +## Inputs + +| Input | Required | Default | Description | +|-------|----------|---------|-------------| +| `grid` | Yes | `qa` | Target grid environment (`qa`, `testing`, or `production`) | +| `version` | Yes | — | Version string to deploy (e.g. the flist version tag) | + +## Steps + +1. **Symlink flist**: Creates a cross-link so that `tf-zos/zos:-3:latest.flist` points to `tf-autobuilder/zos:.flist` + +This is the mechanism used to promote a tested build to a specific grid environment. diff --git a/docs/workflows/publish-bootstrap.md b/docs/workflows/publish-bootstrap.md new file mode 100644 index 000000000..e9390ca89 --- /dev/null +++ b/docs/workflows/publish-bootstrap.md @@ -0,0 +1,20 @@ +# Publish Bootstrap + +**File:** `.github/workflows/publish-bootstrap.yaml` +**Trigger:** Push to files in `bootstrap/bootstrap/**` or changes to the workflow file itself + +## Purpose + +Builds and publishes the ZOS bootstrap binary. The bootstrap is a Rust binary compiled with musl for static linking, which is responsible for downloading and starting all ZOS services on node boot. + +## Steps + +1. **Checkout** the repository +2. **Prepare musl**: Installs `musl` and `musl-tools` for static compilation +3. **Setup Rust toolchain**: Configures stable Rust with `x86_64-unknown-linux-musl` target +4. **Build bootstrap**: Runs `make release` in `bootstrap/bootstrap/` +5. **Collect files**: Copies the built binary to `archive/sbin/bootstrap` +6. **Set name**: Generates a versioned name like `bootstrap-v-dev.flist` +7. **Publish flist**: Uploads to `tf-autobuilder/.flist` +8. **Symlink (development)**: Always creates symlink `bootstrap:development.flist` pointing to the new build +9. **Symlink (release)**: On `main` branch only, also creates symlink `bootstrap:latest.flist` diff --git a/docs/workflows/publish.md b/docs/workflows/publish.md new file mode 100644 index 000000000..a86fe7aed --- /dev/null +++ b/docs/workflows/publish.md @@ -0,0 +1,38 @@ +# Publish (Release) + +**File:** `.github/workflows/publish.yaml` +**Trigger:** Push to any branch, or version tags (`v*`) + +## Purpose + +Builds all ZOS Go binaries (the core daemons) and publishes them as a single flist. This is the main release workflow for ZOS itself (as opposed to `bins.yaml` which handles runtime dependencies). + +## Steps + +1. **Set up Go 1.23** +2. **Checkout** the repository +3. **Build binaries**: Runs `make` in the `cmds/` directory to compile all ZOS daemons +4. **Set tag**: Determines version reference — tag name for releases, short SHA for branches +5. **Set version**: Generates a timestamped version like `v` +6. **Collect files**: Runs `scripts/collect.sh` to gather all binaries and zinit configs into an archive +7. **Publish flist**: Uploads `zos:.flist` to `tf-autobuilder` +8. **Tagging**: On `main`, `testdeploy`, or version tags, creates a tag link `/zos.flist` +9. **Cross tagging (development)**: On `main` or `testdeploy`, cross-tags the build to `tf-zos/development`, making it the latest development release + +## Release Flow + +``` +Push to main + → Build all daemons + → Publish as zos:.flist + → Tag as /zos.flist + → Cross-tag to tf-zos/development (devnet) + +Push tag v* + → Build all daemons + → Publish as zos:.flist + → Tag as /zos.flist + +Manual grid-deploy workflow + → Links tf-zos/zos:-3:latest.flist → tf-autobuilder/zos:.flist +``` diff --git a/docs/workflows/test-bootstrap.md b/docs/workflows/test-bootstrap.md new file mode 100644 index 000000000..2e547674f --- /dev/null +++ b/docs/workflows/test-bootstrap.md @@ -0,0 +1,16 @@ +# Bootstrap Tests + +**File:** `.github/workflows/test-bootstrap.yaml` +**Trigger:** Push to files in `bootstrap/bootstrap/**` or changes to the workflow file itself + +## Purpose + +Runs tests for the ZOS bootstrap binary (Rust) and verifies it compiles successfully. + +## Steps + +1. **Checkout** the repository +2. **Prepare musl**: Installs `musl` and `musl-tools` +3. **Setup Rust toolchain**: Configures stable Rust with `x86_64-unknown-linux-musl` target +4. **Test**: Runs `make test` in `bootstrap/bootstrap/` +5. **Build**: Runs `make release` to verify the release build compiles diff --git a/docs/workflows/test.md b/docs/workflows/test.md new file mode 100644 index 000000000..36772197e --- /dev/null +++ b/docs/workflows/test.md @@ -0,0 +1,19 @@ +# Tests and Coverage + +**File:** `.github/workflows/test.yaml` +**Trigger:** Push to any branch + +## Purpose + +Runs tests and verifies the ZOS Go codebase compiles. Currently the test steps are commented out — only a build verification is performed. + +## Steps + +1. **Set up Go 1.23** +2. **Prepare dependencies**: Installs `libjansson-dev` and `libhiredis-dev` +3. **Checkout** the repository +4. **Build binaries**: Runs `make` in `cmds/` to verify compilation + +## Note + +The actual test execution (`make testrace`) and dependency fetching (`make getdeps`) steps are currently commented out in the workflow. diff --git a/docs/workflows/zos-update-worker-main.md b/docs/workflows/zos-update-worker-main.md new file mode 100644 index 000000000..2612cfc5f --- /dev/null +++ b/docs/workflows/zos-update-worker-main.md @@ -0,0 +1,19 @@ +# ZOS Update Worker (CI) + +**File:** `.github/workflows/zos-update-worker-main.yml` +**Trigger:** +- Push to files in `tools/zos-update-worker/**` +- Pull requests touching `tools/zos-update-worker/**` + +## Purpose + +Runs linting, static analysis, formatting checks, and tests for the ZOS update worker tool. + +## Steps + +1. **Checkout** the repository +2. **Install Go 1.19** +3. **golangci-lint**: Runs the golangci linter with a 3-minute timeout +4. **staticcheck**: Runs static analysis (version 2022.1.3) +5. **gofmt**: Checks for formatting issues +6. **Test**: Runs `go test -v ./...` diff --git a/docs/workflows/zos-update-worker-release.md b/docs/workflows/zos-update-worker-release.md new file mode 100644 index 000000000..bef8bcc63 --- /dev/null +++ b/docs/workflows/zos-update-worker-release.md @@ -0,0 +1,16 @@ +# ZOS Update Worker (Release) + +**File:** `.github/workflows/zos-update-worker-release.yml` +**Trigger:** Push of version tags (`v*`) + +## Purpose + +Builds and publishes the ZOS update worker binary as a GitHub release asset when a new version tag is pushed. + +## Steps + +1. **Checkout** the repository +2. **Install Go 1.19** +3. **Build**: Runs `make build` in `tools/zos-update-worker/` +4. **Get release**: Fetches the GitHub release matching the pushed tag +5. **Upload release asset**: Attaches `bin/zos-update-worker` to the GitHub release