Ai tier ga - #143
Open
kupratyu-splunk wants to merge 213 commits into
Open
Conversation
feat(AIP-3938): AI tier openshift support
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
feat(AIP-4163): Ai pod script enhancements
feat(AIP-4161): Openshift air gap install script
Bring ai-tier-ga in sync with main
… script issues (#126) * feat(openshift): route GPU/CPU workers via ai-tier-node label * resolve codex comments
chore: Merge ai-pod-feature-branch to ai-tier-ga
Creates EC2 infrastructure consumed by k0s_cluster_with_stack.sh: - VPC + public subnet + IGW in a single AZ - Security group with self-referencing rule (private-IP k0s comms) - RHEL 9 instances: controller(s), CPU workers, GPU workers, installer - Installer gets an EIP; k0s nodes stay on private IPs only - Separate EBS volumes for /var/lib/k0s (GPU) and /data/minio (installer) - Auto-creates or reuses AWS key pair, downloads .pem locally - Optional MinIO install via install_minio_ec2.sh after provisioning - Generates my-k0s-config.yaml on the installer with private IPs pre-filled - Commands: provision / output / status / destroy / validate / dry-run - Test config (t3.medium, ~$1-2) for Level 2 validation - Design + test plan in K0S_AWS_PROVISION.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…mount fix
Rewrites CloudFormation-based provisioner to direct AWS CLI calls,
bypassing SCP p-m68tib3s which blocks ec2:RunInstances via CloudFormation.
Key changes:
- IMDSv2 required (HttpTokens=required) on all instances — required by SCP
- EBS encryption on all volumes — required by SCP
- Existing VPC only (vpc-09b191e89c83d588e) — no new VPC creation
- Separate subnets: k0s nodes in private subnet, installer in public subnet with EIP
- SSH ProxyCommand instead of -J flag (fixes key forwarding on macOS)
- mount_disk_via_ssh: mount EBS data disks via SSH after attach, not UserData
(UserData runs before EBS volumes are attached by the provisioner)
- destroy --yes / -y flag for non-interactive teardown
- Fix _INSTANCE grep pattern to include INSTALLER_INSTANCE (was missing)
- Fix ${var,,} bash 3.2 incompatibility with tr [:upper:] [:lower:]
Validated end-to-end: provision → status → output → destroy all pass.
All 4 instances, EBS volumes, EIP, SG, key pair created and destroyed cleanly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…on steps, failure history Updates the provisioner doc to reflect the rewrite from CloudFormation to direct AWS CLI calls. Adds: - SCP compliance table (IMDSv2, EBS encryption, existing VPC requirement) - Corrected architecture diagram (public vs private subnet split) - Full k0s integration guide (Option A from installer, Option B from laptop + ProxyJump) - Development history: 9 documented failures with root causes and fixes (SCP block, VPC, set -e false negative, private subnet EIP, SSH -J key, EBS mount timing, INSTALLER_INSTANCE grep, destroy TTY, bash 3.2 compat) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…s-west-2 VPC When network.vpcId is empty (or omitted in non-us-west-2 regions), the provisioner now creates the full network stack automatically: VPC → IGW → public subnet + route (0.0.0.0/0 → IGW) → NAT GW EIP → NAT GW → private subnet + route (0.0.0.0/0 → NAT GW) All six resources are tagged and tracked in the state file so destroy tears them down cleanly in reverse order (private RT → private subnet → NAT GW → NAT EIP → public RT → public subnet → IGW → VPC). For us-west-2, the default vpcId remains vpc-09b191e89c83d588e (SCP in splunkcloud-ai-dev blocks new-VPC creation; leaving the default avoids breaking existing deployments). Any other region defaults to auto-create. Config additions: network.vpcCidr (default 10.0.0.0/16) network.publicSubnetCidr (default 10.0.1.0/24) network.privateSubnetCidr (default 10.0.2.0/24) Docs updated with Network Modes table, mandatory-fields list, and full config reference with inline comments for each network field. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… to not create AWS resources - Add WARNING comments to ensure_network and destroy_network (untested code path) - Emit runtime warn() when auto-create mode activates - dry-run: skip pick_subnet in auto-create mode (no VPC exists yet) - dry-run: remove ensure_key_pair call (was creating real key pairs in AWS) Dry-run now creates zero AWS resources in both existing-VPC and auto-create modes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…writing
Old behaviour: cat > my-k0s-config.yaml (full overwrite with hardcoded defaults)
New behaviour:
1. If my-k0s-config.yaml exists on installer → back it up as
my-k0s-config.bak-<timestamp>.yaml, then yq-patch in-place.
2. If it doesn't exist → copy k0s-cluster-config.yaml (already
uploaded by setup_installer) as the base, then yq-patch.
Only infrastructure fields are written:
cluster.{name,region,sshKeyPath,sshUser}
nodes.existingIPs.{controllers,workers}
storage.objectStore.{type,bucket,endpoint,auth.*} (when minio.enabled)
All other fields (images, operators, aiPlatform, metallb, ecr, etc.)
are preserved from the user's own file.
IPs are serialised via jq to a proper JSON array before yq injection,
avoiding quoting issues with multi-node configs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: align OpenShift SLIM and RTX timeseries support * fix: trust both in-cluster Splunk issuer aliases * fix: align OpenShift issuer handling with k0s * feat: expose SLIM on a distinct OpenShift NodePort
…k reference Clarify that L40S and H100 GPU worker specs are alternatives, not additive; add the repo clone step to Config Setup to match K0S_README's Quick Start; and drop the redundant macOS brew install snippet. Co-Authored-By: Claude <noreply@anthropic.com>
The GPU worker rows list per-node specs next to a "2 nodes minimum" count, which read as if those figures could be cluster totals. Label the columns and AWS EC2 equivalents as per-node, and remove the now-duplicated summary sentence below the table. Also restore install guidance for macOS/Ubuntu admin workstations, which the prior commit dropped without a replacement. Co-Authored-By: Claude <noreply@anthropic.com>
kubectl and helm aren't in default apt repos, so `apt-get install -y kubectl helm git jq` fails on a fresh Ubuntu 24.04 box, and the yq download needs sudo to write to /usr/local/bin. Add the upstream apt repo for kubectl, use helm's install script, and prefix the yq download/chmod with sudo.
kubectl and helm aren't in RHEL 9's default dnf repos, so replace the non-runnable bullet list with tested install commands (verified in a UBI9 container) alongside the Ubuntu fix from the previous commit (verified in an ubuntu:24.04 container). Pin the RHEL 9 kubectl download to v1.31.2, matching the k0s version already used as the --k0s-version example throughout DEPLOYMENT_GUIDE.md/K0S_README.md/TEST_PLAN.md. airgap_install.sh's K0S_VERSION default was "latest", making air-gapped installs non-reproducible and inconsistent with the pinned version used in every doc example. Default it to v1.31.2+k0s.0 instead; --k0s-version latest still resolves the newest release for anyone who wants it.
Pin yq (v4.44.1) and kubectl (v1.36.1) to versions the repo already relies on instead of "latest", add the missing dnf-plugins-core dependency before dnf config-manager on RHEL9, standardize the k0s default version (v1.36.1+k0s.0) across airgap_install.sh and docs instead of vague "latest" phrasing, document the docker group setup step, and add branch-specific clone/download instructions. Co-Authored-By: Claude <noreply@anthropic.com>
Move the detailed RHEL9 install commands out of the quick-reference doc into K0S_README.md's Required Tools section (alongside the existing macOS/Ubuntu blocks), and merge the quick-reference doc's separate Ubuntu/RHEL9 callouts into one summary with a bullet list of what each tool needs, linking to K0S_README.md for exact commands. Co-Authored-By: Claude <noreply@anthropic.com>
…0s-clarify-gpu-hw # Conflicts: # docs/deployment/k0s-quick-reference.md
The combined Ubuntu/RHEL9 tool bullet list implied dnf-plugins-core and the dnf-based git/jq install applied to both OSes; dnf is RHEL-only, and Docker is only needed there too, matching K0S_README.md's actual per-OS instructions. Co-Authored-By: Claude <noreply@anthropic.com>
* fix(k0s): preserve registered node names * fix(k0s): validate hostname fallbacks * fix(k0s): wait for late node registration
Ubuntu admins had no path to install Docker or crane despite the mandatory image-mirroring steps requiring one of them; document crane (cross-platform, no daemon/root needed) as the default and Docker as an alternative on both Ubuntu and RHEL 9. Also parameterize the mirroring loops with a TAG variable instead of hardcoding "preview".
Docs/k0s clarify gpu hw
Admins whose browser only has SSH access to a bastion/installer machine had no documented way to reach Splunk Web or the SAIA API. Add a SOCKS-tunnel walkthrough to K0S_README.md and link it from the quick-reference guide.
Derive the Splunk port-forward target from NAMESPACE/STANDALONE_NAME instead of hardcoding the default service name, and add a Linux (google-chrome) command alongside the macOS-only open -na example. Co-Authored-By: Claude <noreply@anthropic.com>
docs: document SSH SOCKS tunnel for accessing internal Splunk/SAIA
fix(saia): roll v2 workloads on issuer changes
fix(ai-tier): pin Ray 2.56 and disable BiEncoder
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Related Issues
Type of Change
Changes Made
Testing Performed
make test)make lint)Test Environment
Test Steps
Documentation
Checklist
Breaking Changes
Impact:
Migration Path:
Screenshots/Recordings
Additional Notes
Reviewer Notes
Please pay special attention to:
Commit Message Convention: This PR follows Conventional Commits