Add kubernetes events

[yairpod]

Emitting kubernetes events on major registration/attestation flow points.
This will allow cluster admins to follow and debug what happens in confidential clusters.

Summary by Sourcery

Add Kubernetes event reporting throughout the confidential-cluster registration and attestation workflows to improve operational visibility and debugging.

New Features:

• Emit Kubernetes events for key registration, machine registration, attestation-key approval, key provisioning and revocation, and reference-value computation flows.
• Provide event polling utilities and integration coverage for validating emitted registration and attestation events.

Enhancements:

• Centralize Kubernetes event recording and controller reporter setup for reusable event publication across services and controllers.

Tests:

• Add end-to-end verification of expected Kubernetes events across machine registration, attestation, computation, provisioning, and cleanup flows.

Chores:

• Grant the operator permission to create and patch Kubernetes events.

[sourcery-ai]

Reviewer's Guide

This PR adds Kubernetes event emission across the registration, attestation, key provisioning, and reference value computation flows, introduces a shared event-recording helper, wires recorders into controllers and HTTP services, and adds tests and utilities to validate the new events end‑to‑end.

Sequence diagram for machine registration events

sequenceDiagram
    participant User
    participant RegisterServer
    participant KubernetesAPI
    participant Recorder
    participant Machine

    User->>RegisterServer: GET register endpoint
    RegisterServer->>KubernetesAPI: create Machine
    KubernetesAPI-->>RegisterServer: created Machine
    RegisterServer->>Recorder: record_event MachineRegistered
    Recorder->>KubernetesAPI: publish Event for Machine

Loading

Sequence diagram for attestation key registration and approval events

sequenceDiagram
    participant Client
    participant KeyRegister as attestation-key-register
    participant KubernetesAPI
    participant Recorder
    participant AKController as ak-controller
    participant Machine
    participant AttestationKey

    Client->>KeyRegister: PUT attestation key
    KeyRegister->>KubernetesAPI: create AttestationKey
    KubernetesAPI-->>KeyRegister: created AttestationKey
    KeyRegister->>Recorder: record_event AttestationKeyRegistered
    Recorder->>KubernetesAPI: publish Event for AttestationKey
    AKController->>Recorder: record_event AttestationKeyApproved
    Recorder->>KubernetesAPI: publish Event for AttestationKey
    AKController->>Recorder: record_event AttestationKeyApproved
    Recorder->>KubernetesAPI: publish Event for Machine

Loading

Sequence diagram for key provisioning events

sequenceDiagram
    participant MachineController as keygen-controller
    participant KubernetesAPI
    participant Trustee
    participant Recorder
    participant Machine

    MachineController->>Trustee: generate_secret
    MachineController->>Trustee: send_secret
    Trustee-->>MachineController: provisioning result
    alt provisioning succeeds
        MachineController->>Recorder: record_event KeyProvisioned
        Recorder->>KubernetesAPI: publish Event for Machine
    else provisioning fails
        MachineController->>Recorder: record_event KeyProvisioningFailed
        Recorder->>KubernetesAPI: publish Warning Event for Machine
    end

Loading

Sequence diagram for reference value computation events

sequenceDiagram
    participant ImageController as rv-controller
    participant KubernetesAPI
    participant ComputationJob
    participant Recorder
    participant ApprovedImage

    ImageController->>ImageController: handle_new_image
    ImageController->>Recorder: record_event ComputationStarted
    Recorder->>KubernetesAPI: publish Event for ApprovedImage
    ComputationJob->>ImageController: job_reconcile
    ImageController->>KubernetesAPI: delete completed Job
    ImageController->>Recorder: record_event ComputationCompleted
    Recorder->>KubernetesAPI: publish Event for ApprovedImage

Loading

File-Level Changes

Change	Details	Files
Introduce a reusable helper for publishing Kubernetes events and wire in logging support.	Add log as a workspace dependency in the shared library to support event publication warnings. Add record_event async helper wrapping kube runtime event Recorder and k8s_openapi ObjectReference / Event types. Keep helper tolerant of absence of a Recorder (Option) and log a warning on publish failure instead of failing the controller logic.	lib/Cargo.toml lib/src/lib.rs
Emit events during attestation key registration via the attestation-key-register HTTP service.	Introduce AppState to hold both kube Client and Recorder and switch axum handlers to use this state instead of raw Client. Create a Reporter/Recorder for the attestation-key-register service and pass it through Router state. On duplicate-key detection, publish a Warning event 'DuplicateKeyRejected' referencing the existing AttestationKey. On successful AttestationKey creation, publish a Normal event 'AttestationKeyRegistered' referencing the new AttestationKey.	attestation-key-register/src/main.rs
Emit events when machines are registered via the register-server HTTP service.	Refactor register-server to use an AppState with Client and Recorder instead of bare Client state. Construct a Reporter/Recorder for the register-server controller and store it in application state. Change create_machine to return the created Machine resource so it can be used as the event’s regarding object. After successfully creating a Machine, publish a Normal 'MachineRegistered' event referencing the Machine.	register-server/src/main.rs
Add event recording to operator controllers for attestation-key approval, key provisioning/revocation, and reference value computation, while standardizing controller context with a Recorder.	Extend AkContextData with an optional Recorder, initializing it via a new operator::new_recorder helper. On attestation key approval, emit paired Normal 'AttestationKeyApproved' events: one regarding the AttestationKey and one regarding the Machine. Introduce ControllerContext struct (client + optional Recorder) and use it in reference_values and register_server controllers instead of passing Arc<Client directly. In reference_values job_reconcile, emit Normal 'ComputationCompleted' events regarding ApprovedImage owners once reference values are updated. In reference_values image_add_reconcile, emit Normal 'ComputationStarted' when PCR computation begins and Warning 'ComputationFailed' when it fails, both regarding the ApprovedImage. In register_server keygen_reconcile, emit Normal 'KeyProvisioned' on successful key generation and send, Warning 'KeyProvisioningFailed' on failure, and Normal 'KeyRevoked' when decryption keys are deleted during cleanup. Update controller launch functions (rv_job, rv_image, keygen) to construct ControllerContext with Recorder via new_recorder and pass that context into Controller::run. Adjust tests to use the new ControllerContext abstraction instead of Arc<Client.	operator/src/lib.rs operator/src/attestation_key_register.rs operator/src/reference_values.rs operator/src/register_server.rs
Extend RBAC and test utilities to support and validate event emission, and add an integration test for the complete event flow.	Update kubebuilder RBAC annotations to grant create/patch permissions on events.k8s.io events. Add wait_for_event utility using Api, ListParams, and Poller to poll for events by regarding resource name and reason with a timeout. Add attestation integration test that runs a VM, performs attestation, discovers Machine, AttestationKey, and ApprovedImage resources, and waits for eight specific events: ComputationStarted, ComputationCompleted, MachineRegistered, AttestationKeyRegistered, AttestationKeyApproved (on AK and Machine), KeyProvisioned, and KeyRevoked. Use existing wait_for_resource_deleted helper to trigger and then verify KeyRevoked via event after Machine deletion.	api/v1alpha1/crds.go test_utils/src/lib.rs tests/attestation.rs

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 1 issue

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="api/v1alpha1/crds.go" line_range="36" />
<code_context>
 // +kubebuilder:rbac:groups=trusted-execution-clusters.io,resources=trustedexecutionclusters;machines;approvedimages;attestationkeys,verbs=create;delete;get;list;patch;update;watch
 // +kubebuilder:rbac:groups=trusted-execution-clusters.io,resources=trustedexecutionclusters/finalizers;machines/finalizers;attestationkeys/finalizers;approvedimages/finalizers,verbs=update
 // +kubebuilder:rbac:groups=trusted-execution-clusters.io,resources=trustedexecutionclusters/status;machines/status;approvedimages/status;attestationkeys/status,verbs=get;patch;update
+// +kubebuilder:rbac:groups=events.k8s.io,resources=events,verbs=create;patch

 // TrustedExecutionClusterSpec defines the desired state of TrustedExecutionCluster
</code_context>
<issue_to_address>
**issue (bug_risk):** The event recorder publishes `events.k8s.io` Events, but the checked-in operator RBAC grants `create;patch` only for core `events` (`apiGroups: [""]`), not `events.k8s.io`. Every `record_event` call therefore receives a Kubernetes authorization error in deployed clusters, which is only logged and leaves the new events absent.

**Triggers:** When the checked-in RBAC manifests are deployed without regenerating them to add the `events.k8s.io` rule.

**Suggested fix:** Add `apiGroups: ["events.k8s.io"]` with `resources: ["events"]` and `verbs: ["create", "patch"]` to the deployed operator RBAC, and regenerate all packaged manifests.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 1 finding to address first, and if the event logic is wrong, Kubernetes Event objects can be emitted with incorrect or overly frequent messages and remain after the code is reverted, though they are bounded and can be deleted. The added RBAC grant also changes what these workloads may write in the cluster, but it does not grant access to application data or alter the underlying provisioning decisions.

Blocking findings: api/v1alpha1/crds.go:36

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[iroykaufman]

Why not use cache(aka AkContextData) here insted of client?

[Jakob-Naucke]

NB this would change a bit with #330

[yairpod]

AkContextData is Holds 4 reflector Stores, these are not needed where ControllerContext is used,
It will be a waste to create them and not use them.
We could change AkContextData to extend ControllerContext as it hold 2 of the 6 fields of AkContextData, but I don't think it's worth it.

[iroykaufman]

I see, but after extending the use of the reflector, this will need to be changed (probably after #330 gets in). For example, here we can use the reflector.

[Jakob-Naucke]

NB this would change a bit with #330

[yairpod]

/retest

[Jakob-Naucke]

the assumption was that only CI networks were unstable enough to actually witness such events, where this is instead handled by timeout multipliers. did you observe this elsewhere?

[yairpod]

Some delays were witnessed in the integration tests.
The timeout multipliers was added to handle these isses.

[Jakob-Naucke]

Hmm. From GHA on this PR:

2026-09-02T07:31:40Z INFO: test_attestation_events: Event ComputationStarted verified
2026-09-02T07:31:40Z INFO: test_attestation_events: Event ComputationCompleted verified

I'd rather keep this at 60 (keeping the CI multiplier) and see if it is a real problem.

On a different note, the multiplier in #330 assumes all timeouts to be at least 60 seconds so that there can be a retry with 5 minutes read timeout and a multiplier of 6. If that remains to be how we work, we could enforce that better, but just for this PR, would you mind setting all to 60?

[yairpod]

I will test that it works well on my testing env (which has high INTEGRATION_TEST_THREADS capabilitys and finds many testing race conditions), if it passes i will change it

[Jakob-Naucke]

thanks for the update! with regards to my last paragraph, could we keep all timeout bases at 60 for now?

[yairpod]

By all "timeout bases" do you mean just all timeout numbers (before scaling) in the tests?

[Jakob-Naucke]

yes

[yairpod]

Sure, Should I start with the rest of the timeouts in this PR?

[Jakob-Naucke]

yes, all existing timeout bases are 60 or greater afaict

[yairpod]

Done

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Jakob-Naucke, yairpod

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[yairpod]

/test azure-integration-test

[Jakob-Naucke]

@yairpod your first commit isn't signed, could you check on that?

btw the Azure run that just failed had this in the Trustee log:

2026-09-03T11:52:24.106272Z DEBUG Azure vTPM SEV-SNP: verifier::az_snp_vtpm: SNP report_data verification completed successfully
2026-09-03T11:52:24.106450Z ERROR kbs::error: AttestationError(RcarAttestFailed { source: verify TEE evidence failed

Caused by:
    Verifier evaluate failed: Failed to get CPU Generation
    
    Caused by:
        Attestation report version 3+ is missing CPU family ID })

which I haven't seen before, but is not likely to be your fault

[openshift-ci]

New changes are detected. LGTM label has been removed.

[yairpod]

@yairpod your first commit isn't signed, could you check on that?

Vary strange, but fixed