docs for workflows

docs/workflows/bin-package-18.04.md

@@ -0,0 +1,29 @@
+# bin-package-18.04
+
+**File:** `.github/workflows/bin-package-18.04.yaml`
+**Trigger:** Called by other workflows (`workflow_call`)
+
+## Purpose
+
+Builds a single runtime package using an **Ubuntu 18.04** container. This is identical to `bin-package` but forces the older Ubuntu version for packages that require it (e.g. `mdadm`).
+
+## Inputs
+
+| Input | Required | Description |
+|-------|----------|-------------|
+| `package` | Yes | Name of the package to build |
+
+## Secrets
+
+| Secret | Description |
+|--------|-------------|
+| `token` | Hub JWT token for publishing flists |
+
+## Steps
+
+1. **Checkout** the repository
+2. **Set tag**: Determines the version reference — uses git tag name for tag pushes, or short SHA for branch pushes
+3. **Setup basesystem**: Runs `bins/bins-extra.sh --package basesystem` to install build prerequisites
+4. **Build package**: Runs `bins/bins-extra.sh --package <package>` to compile the binary
+5. **Publish flist**: Uploads the built binary as an flist to `tf-autobuilder/<package-name>.flist` on the hub
+6. **Tagging**: On `main`, `testdeploy`, or version tags (`v*`), creates a tag link `<reference>/<package>.flist` pointing to the published flist

docs/workflows/bin-package-no-tag.md

@@ -0,0 +1,29 @@
+# bin-package-no-tag
+
+**File:** `.github/workflows/bin-package-no-tag.yaml`
+**Trigger:** Called by other workflows (`workflow_call`)
+
+## Purpose
+
+Builds a single runtime package using Ubuntu 20.04 but **never tags** the built binary. This means any package built with this workflow never becomes part of a ZOS installation release. Used for packages like `qsfs` and `traefik` that are published independently.
+
+## Inputs
+
+| Input | Required | Description |
+|-------|----------|-------------|
+| `package` | Yes | Name of the package to build |
+
+## Secrets
+
+| Secret | Description |
+|--------|-------------|
+| `token` | Hub JWT token for publishing flists |
+
+## Steps
+
+1. **Checkout** the repository
+2. **Setup basesystem**: Installs build prerequisites via `bins/bins-extra.sh --package basesystem`
+3. **Build package**: Compiles the binary via `bins/bins-extra.sh --package <package>`
+4. **Publish flist**: Uploads the built binary as an flist to `tf-autobuilder/<package-name>.flist` on the hub
+
+No tagging step is performed — the flist is published but not linked to any release version.

docs/workflows/bin-package.md

@@ -0,0 +1,29 @@
+# bin-package
+
+**File:** `.github/workflows/bin-package.yaml`
+**Trigger:** Called by other workflows (`workflow_call`)
+
+## Purpose
+
+Builds a single runtime package using Ubuntu 20.04. The built binary is always published to `tf-autobuilder` and then tagged with the release version or the short SHA of the commit head (on main branch). This is the standard build workflow invoked by `bins.yaml` for most packages.
+
+## Inputs
+
+| Input | Required | Description |
+|-------|----------|-------------|
+| `package` | Yes | Name of the package to build |
+
+## Secrets
+
+| Secret | Description |
+|--------|-------------|
+| `token` | Hub JWT token for publishing flists |
+
+## Steps
+
+1. **Checkout** the repository
+2. **Set tag**: Determines the version reference — uses git tag name for tag pushes (e.g. `v1.2.3`), or short SHA for branch pushes
+3. **Setup basesystem**: Runs `apt update` then `bins/bins-extra.sh --package basesystem`
+4. **Build package**: Compiles the binary via `bins/bins-extra.sh --package <package>`
+5. **Publish flist**: On `main`, `testdeploy`, or version tags, uploads to `tf-autobuilder/<package-name>.flist`
+6. **Tagging**: On `main`, `testdeploy`, or version tags, creates a tag link `<reference>/<package>.flist` pointing to the published flist

docs/workflows/bins.md

@@ -0,0 +1,51 @@
+# bins (Runtime Packages)
+
+**File:** `.github/workflows/bins.yaml`
+**Trigger:** Push to any branch, or version tags (`v*`)
+
+## Purpose
+
+Orchestrates the build of **all runtime packages** that ZOS depends on. Each package is built in parallel by calling one of the reusable workflows:
+
+- `bin-package.yaml` — standard build + tag (most packages)
+- `bin-package-18.04.yaml` — build on Ubuntu 18.04 (for packages requiring older glibc)
+- `bin-package-no-tag.yaml` — build without tagging (packages managed independently)
+
+## Packages
+
+### Standard packages (bin-package)
+
+| Package | Description |
+|---------|-------------|
+| `containerd` | Container runtime |
+| `runc` | OCI runtime |
+| `virtwhat` | Hypervisor detection |
+| `yggdrasil` | Overlay network daemon |
+| `rfs` | Remote filesystem |
+| `hdparm` | Disk parameters tool |
+| `corex` | Container interactive shell |
+| `shimlogs` | Container log shim |
+| `cloudhypervisor` | VM hypervisor |
+| `tailstream` | Log tail streaming |
+| `virtiofsd` | Virtio filesystem daemon |
+| `vector` | Log/metrics collection |
+| `nnc` | Network connectivity checker |
+| `lshw` | Hardware lister |
+| `cloudconsole` | Cloud console |
+| `misc` | Miscellaneous tools |
+| `iperf` | Network performance |
+| `cpubench` | CPU benchmarking |
+| `mycelium` | Mycelium network daemon |
+
+### Ubuntu 18.04 packages (bin-package-18.04)
+
+| Package | Description |
+|---------|-------------|
+| `mdadm` | RAID management |
+
+### No-tag packages (bin-package-no-tag)
+
+| Package | Description |
+|---------|-------------|
+| `qsfs` | Quantum Safe File System |
+| `traefik` | Reverse proxy |

docs/workflows/codeql-analysis.md

@@ -0,0 +1,18 @@
+# CodeQL Analysis
+
+**File:** `.github/workflows/codeql-analysis.yml`
+**Trigger:**
+- Push to `main`
+- Pull requests targeting `main`
+- Scheduled: every Wednesday at 11:00 UTC
+
+## Purpose
+
+Runs GitHub's CodeQL static analysis tool on the Go codebase to find security vulnerabilities and code quality issues.
+
+## Steps
+
+1. **Checkout** the repository (with depth 2 for PR head detection)
+2. **Initialize CodeQL** for Go language scanning
+3. **Autobuild** — CodeQL automatically detects and builds the Go project
+4. **Perform analysis** — runs the CodeQL queries and reports findings

docs/workflows/grid-deploy.md

@@ -0,0 +1,21 @@
+# Grid Deploy
+
+**File:** `.github/workflows/grid-deploy.yaml`
+**Trigger:** Manual dispatch (`workflow_dispatch`)
+
+## Purpose
+
+Deploys a specific ZOS version to a target grid environment (qa, testing, or production) by creating a cross-link on the hub. This effectively makes the specified version the "latest" for that grid.
+
+## Inputs
+
+| Input | Required | Default | Description |
+|-------|----------|---------|-------------|
+| `grid` | Yes | `qa` | Target grid environment (`qa`, `testing`, or `production`) |
+| `version` | Yes | — | Version string to deploy (e.g. the flist version tag) |
+
+## Steps
+
+1. **Symlink flist**: Creates a cross-link so that `tf-zos/zos:<grid>-3:latest.flist` points to `tf-autobuilder/zos:<version>.flist`
+
+This is the mechanism used to promote a tested build to a specific grid environment.

docs/workflows/publish-bootstrap.md

@@ -0,0 +1,20 @@
+# Publish Bootstrap
+
+**File:** `.github/workflows/publish-bootstrap.yaml`
+**Trigger:** Push to files in `bootstrap/bootstrap/**` or changes to the workflow file itself
+
+## Purpose
+
+Builds and publishes the ZOS bootstrap binary. The bootstrap is a Rust binary compiled with musl for static linking, which is responsible for downloading and starting all ZOS services on node boot.
+
+## Steps
+
+1. **Checkout** the repository
+2. **Prepare musl**: Installs `musl` and `musl-tools` for static compilation
+3. **Setup Rust toolchain**: Configures stable Rust with `x86_64-unknown-linux-musl` target
+4. **Build bootstrap**: Runs `make release` in `bootstrap/bootstrap/`
+5. **Collect files**: Copies the built binary to `archive/sbin/bootstrap`
+6. **Set name**: Generates a versioned name like `bootstrap-v<YYMMDD.HHMMSS.0>-dev.flist`
+7. **Publish flist**: Uploads to `tf-autobuilder/<name>.flist`
+8. **Symlink (development)**: Always creates symlink `bootstrap:development.flist` pointing to the new build
+9. **Symlink (release)**: On `main` branch only, also creates symlink `bootstrap:latest.flist`

docs/workflows/publish.md

@@ -0,0 +1,38 @@
+# Publish (Release)
+
+**File:** `.github/workflows/publish.yaml`
+**Trigger:** Push to any branch, or version tags (`v*`)
+
+## Purpose
+
+Builds all ZOS Go binaries (the core daemons) and publishes them as a single flist. This is the main release workflow for ZOS itself (as opposed to `bins.yaml` which handles runtime dependencies).
+
+## Steps
+
+1. **Set up Go 1.23**
+2. **Checkout** the repository
+3. **Build binaries**: Runs `make` in the `cmds/` directory to compile all ZOS daemons
+4. **Set tag**: Determines version reference — tag name for releases, short SHA for branches
+5. **Set version**: Generates a timestamped version like `v<YYMMDD.HHMMSS.0>`
+6. **Collect files**: Runs `scripts/collect.sh` to gather all binaries and zinit configs into an archive
+7. **Publish flist**: Uploads `zos:<version>.flist` to `tf-autobuilder`
+8. **Tagging**: On `main`, `testdeploy`, or version tags, creates a tag link `<reference>/zos.flist`
+9. **Cross tagging (development)**: On `main` or `testdeploy`, cross-tags the build to `tf-zos/development`, making it the latest development release
+
+## Release Flow
+
+```
+Push to main
+  → Build all daemons
+  → Publish as zos:<version>.flist
+  → Tag as <short-sha>/zos.flist
+  → Cross-tag to tf-zos/development (devnet)
+
+Push tag v*
+  → Build all daemons
+  → Publish as zos:<version>.flist
+  → Tag as <tag>/zos.flist
+
+Manual grid-deploy workflow
+  → Links tf-zos/zos:<grid>-3:latest.flist → tf-autobuilder/zos:<version>.flist
+```

docs/workflows/test-bootstrap.md

@@ -0,0 +1,16 @@
+# Bootstrap Tests
+
+**File:** `.github/workflows/test-bootstrap.yaml`
+**Trigger:** Push to files in `bootstrap/bootstrap/**` or changes to the workflow file itself
+
+## Purpose
+
+Runs tests for the ZOS bootstrap binary (Rust) and verifies it compiles successfully.
+
+## Steps
+
+1. **Checkout** the repository
+2. **Prepare musl**: Installs `musl` and `musl-tools`
+3. **Setup Rust toolchain**: Configures stable Rust with `x86_64-unknown-linux-musl` target
+4. **Test**: Runs `make test` in `bootstrap/bootstrap/`
+5. **Build**: Runs `make release` to verify the release build compiles

docs/workflows/test.md

@@ -0,0 +1,19 @@
+# Tests and Coverage
+
+**File:** `.github/workflows/test.yaml`
+**Trigger:** Push to any branch
+
+## Purpose
+
+Runs tests and verifies the ZOS Go codebase compiles. Currently the test steps are commented out — only a build verification is performed.
+
+## Steps
+
+1. **Set up Go 1.23**
+2. **Prepare dependencies**: Installs `libjansson-dev` and `libhiredis-dev`
+3. **Checkout** the repository
+4. **Build binaries**: Runs `make` in `cmds/` to verify compilation
+
+## Note
+
+The actual test execution (`make testrace`) and dependency fetching (`make getdeps`) steps are currently commented out in the workflow.

docs/workflows/zos-update-worker-main.md

@@ -0,0 +1,19 @@
+# ZOS Update Worker (CI)
+
+**File:** `.github/workflows/zos-update-worker-main.yml`
+**Trigger:**
+- Push to files in `tools/zos-update-worker/**`
+- Pull requests touching `tools/zos-update-worker/**`
+
+## Purpose
+
+Runs linting, static analysis, formatting checks, and tests for the ZOS update worker tool.
+
+## Steps
+
+1. **Checkout** the repository
+2. **Install Go 1.19**
+3. **golangci-lint**: Runs the golangci linter with a 3-minute timeout
+4. **staticcheck**: Runs static analysis (version 2022.1.3)
+5. **gofmt**: Checks for formatting issues
+6. **Test**: Runs `go test -v ./...`

docs/workflows/zos-update-worker-release.md

@@ -0,0 +1,16 @@
+# ZOS Update Worker (Release)
+
+**File:** `.github/workflows/zos-update-worker-release.yml`
+**Trigger:** Push of version tags (`v*`)
+
+## Purpose
+
+Builds and publishes the ZOS update worker binary as a GitHub release asset when a new version tag is pushed.
+
+## Steps
+
+1. **Checkout** the repository
+2. **Install Go 1.19**
+3. **Build**: Runs `make build` in `tools/zos-update-worker/`
+4. **Get release**: Fetches the GitHub release matching the pushed tag
+5. **Upload release asset**: Attaches `bin/zos-update-worker` to the GitHub release