Skip to content

Add cert-manager 1.13.0 to 1.14.x GoNoGo upgrade bundle - #201

Open
Azahorscak wants to merge 5 commits into
mainfrom
cursor/cert-manager-1.13-to-1.14-gonogo-bundle-242b
Open

Add cert-manager 1.13.0 to 1.14.x GoNoGo upgrade bundle#201
Azahorscak wants to merge 5 commits into
mainfrom
cursor/cert-manager-1.13-to-1.14-gonogo-bundle-242b

Conversation

@Azahorscak

Copy link
Copy Markdown
Member

Description

What's the goal of this PR?

This PR adds a comprehensive GoNoGo bundle for assessing cert-manager upgrades from version 1.13.0 to 1.14.x, along with a detailed upgrade assessment document to help users make informed decisions about the upgrade.

What changes did you make?

  1. New GoNoGo Bundle (pkg/bundle/bundles/cert-manager.yaml):

    • Defines version range 1.13.0 → 1.14.6
    • Kubernetes compatibility checks (1.24-1.31)
    • Required API version validation
    • Three OPA policy checks:
      • Detects deprecated certmanager.k8s.io/* annotations on Ingress resources
      • Detects deprecated certmanager.k8s.io/* annotations on Certificate resources
      • Identifies outdated startupapicheck jobs using old ctl image
    • Comprehensive warnings about version-specific bugs
    • Resource scanning for ingresses, secrets, certificates, issuers, etc.
  2. Upgrade Assessment Document (CERT_MANAGER_1.13_TO_1.14_GONOGO.md):

    • Executive summary with clear GO/NO-GO recommendation
    • Kubernetes version compatibility matrix
    • Detailed breakdown of breaking changes with severity levels
    • Known issues in specific versions (v1.14.0-v1.14.3)
    • Pre-upgrade checklist
    • Step-by-step upgrade and rollback procedures
    • Risk assessment matrix
    • Usage instructions for the GoNoGo bundle

Key Findings

Recommendation: GO with caution ⚠️

The upgrade is feasible and recommended (cert-manager 1.13.0 reached EOL on June 5, 2024), but with critical considerations:

  • CRITICAL: Skip versions 1.14.0, 1.14.1, 1.14.2, and 1.14.3 due to known bugs
  • Target version: 1.14.4 or later (recommend v1.14.6)
  • Breaking changes:
    1. New startupapicheck OCI image (impacts air-gapped environments)
    2. KeyUsage/BasicConstraints now encoded as critical in CSR
    3. ACME HTTP01 solver pods include new cluster-autoscaler annotation

What alternative solution should we consider, if any?

Alternative approaches considered:

  • Manual upgrade assessment: Less systematic, prone to missing issues
  • Direct testing without bundle: Reactive rather than proactive
  • Skipping OPA checks: Would miss deprecated annotations in existing resources

The GoNoGo bundle approach provides automated, repeatable validation that can be integrated into CI/CD pipelines and helps teams assess upgrade readiness before attempting the upgrade.

Additional Context

This bundle follows the established pattern from existing bundles (metrics-server, aws-load-balancer-controller) and is based on:

  • Official cert-manager upgrade documentation
  • cert-manager release notes (v1.13.x and v1.14.x series)
  • Known issues documented by the cert-manager team
  • Kubernetes version compatibility matrix

The bundle can be used by running:

gonogo check -b pkg/bundle/bundles/cert-manager.yaml

Checklist

  • I have signed the CLA
  • I have updated/added any relevant documentation

Slack Thread

Open in Web Open in Cursor 

- Add comprehensive GoNoGo bundle for cert-manager upgrade from 1.13.0 to 1.14.x
- Include OPA checks for deprecated annotations and startupapicheck image validation
- Add detailed upgrade assessment document with risk analysis and recommendations
- Document breaking changes, known issues, and version-specific guidance
- Provide pre-upgrade checklist and upgrade procedures
- Recommend target version 1.14.4+ to avoid known bugs in earlier 1.14.x releases

Ref: https://cert-manager.io/docs/releases/upgrading/upgrading-1.13-1.14/

Co-authored-by: Adam Zahorscak <Azahorscak@users.noreply.github.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

cursoragent and others added 4 commits May 1, 2026 15:56
- Add SUMMARY.md with concise upgrade decision matrix
- Include critical requirements and version constraints
- Provide quick command reference for upgrade and rollback
- Summarize risk levels and timeline
- Link to comprehensive assessment and bundle files

Co-authored-by: Adam Zahorscak <Azahorscak@users.noreply.github.com>
- Add comprehensive decision flow diagram for upgrade path
- Include version selection matrix by Kubernetes version
- Provide risk assessment for different scenarios
- Add action priority matrix with ownership
- Document common pitfalls and best practices
- Include emergency rollback decision tree
- Add quick command cheat sheet for common operations

Co-authored-by: Adam Zahorscak <Azahorscak@users.noreply.github.com>
- Add comprehensive usage guide for cert-manager bundle
- Document all OPA checks with examples and remediation steps
- Include severity levels and interpretation guide
- Provide CI/CD integration examples (GitHub Actions)
- Add remediation examples with before/after YAML
- Include pre-upgrade checklist
- Link to all related documentation files

Co-authored-by: Adam Zahorscak <Azahorscak@users.noreply.github.com>
- Add navigation guide for all assessment documents
- Include quick reference tables for versions and breaking changes
- Provide use case-based reading guides
- Add command examples for running assessments
- Include metadata and success criteria checklist
- Link to all internal and external resources

Co-authored-by: Adam Zahorscak <Azahorscak@users.noreply.github.com>
@fairwinds-insights

Copy link
Copy Markdown

Fairwinds Insights Scan Results

View the full report at insights.fairwinds.com.

✅ No new Action Items detected!

1 similar comment
@fairwinds-insights

Copy link
Copy Markdown

Fairwinds Insights Scan Results

View the full report at insights.fairwinds.com.

✅ No new Action Items detected!

@Azahorscak
Azahorscak marked this pull request as ready for review May 1, 2026 16:05
@transient1

Copy link
Copy Markdown
Contributor

I almost wonder if we want these additional files here? The decision tree, summary, ettc.

@Azahorscak

Copy link
Copy Markdown
Member Author

Yeah i'm unsure. I'll review the others and see if the pattern is clear or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants