Skip to content

Real-time progress output during long-running test execution #5

@suin

Description

@suin

Background

Kubernetes E2E tests can take a long time due to resource creation, waiting for Ready state, retries, etc. Currently, kest is designed as a "post-hoc diagnostic" tool — reports and events are only output after test completion.

Problem

During test execution, the only thing visible to the user is Bun's test name [pass/fail] Xms. There is no real-time feedback at all while a test is running.

What's invisible during execution:

Situation Current behavior
kubectl apply running stdout/stderr are piped — nothing is displayed
retryUntil waiting for a resource to become Ready (e.g. 60s) Process appears frozen
Errors during retry attempts Silently swallowed, moves to next retry
cleanup/revert phase Same silence

When a long-running test times out, there's no way to know where it was stuck in real-time. You only find out after completion by reading the report.

Expected behavior

Real-time visibility into what is happening during test execution. For example:

✓ Create Namespace kest-abc12
✓ Apply Deployment "my-app"
⟳ Assert Deployment "my-app" ... retrying (attempt 3/-, elapsed 4.2s)

Information that would be useful to see in real-time:

  • Currently executing action (apply, assert, delete, etc.)
  • Whether a retry is in progress, which attempt, and elapsed time
  • Success/failure of each action

Technical context

The current Recorder accumulates events purely in-memory with no streaming output mechanism. The following events are already being recorded, so a real-time output layer on top of these could make this possible:

  • ActionStart / ActionEnd
  • RetryStart / RetryAttempt / RetryEnd
  • RevertingsStart / RevertingsEnd

Considerations

  • Should this be enabled by default, or opt-in via an environment variable (e.g. KEST_VERBOSE=1) or option?
  • Output granularity: action level / command level / per-retry-attempt level
  • Output format depending on environment: plain text for CI, spinner/colors for interactive terminals
  • Coexistence with Bun's test runner output

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions