Skip to content

Add end-to-end upload-pipeline integration test harness#109

Open
dobby-coder[bot] wants to merge 2 commits intomainfrom
integration-test-harness
Open

Add end-to-end upload-pipeline integration test harness#109
dobby-coder[bot] wants to merge 2 commits intomainfrom
integration-test-harness

Conversation

@dobby-coder
Copy link
Copy Markdown
Contributor

@dobby-coder dobby-coder Bot commented Apr 24, 2026

Summary

Adds the integration test harness requested in #108. Boots a real Rocket instance via a new build_rocket(figment, vk) builder and drives the full POST /fileupload/initPUT /fileupload/<uuid>POST /fileupload/finalize/<uuid> flow, using payloads sealed with pg-core's streaming Sealer so upload_finalize's real Unsealer runs.

What changed

  • src/main.rs — extract build_rocket(figment, vk) and default_figment() from the #[launch] rocket() entrypoint. Production launch is unchanged (still fetches VerifyingKey via minreq); tests inject their own.
  • src/config.rs / src/email.rs — new optional email_stub: bool config field (default false). When true, send_email logs and returns Ok without touching SMTP. Lets finalize's happy path run in tests without a mail server; also useful for local dev.
  • Cargo.toml — add pg-core with the test feature and a rand 0.8 alias as dev-dependencies only (pg-core's APIs use rand 0.8 while the crate itself uses 0.9).
  • src/main.rs — new integration test module (9 tests):
    • upload_happy_path_init_chunk_finalize — single-chunk round-trip, asserts 200 OK
    • upload_happy_path_multi_chunk — two-chunk round-trip (~2 MiB) exercising the rolling token chain
    • upload_init_rejects_invalid_email
    • upload_chunk_rejects_wrong_cryptify_token
    • upload_chunk_unknown_uuid_returns_404
    • upload_chunk_rejects_content_range_misalignment
    • upload_finalize_rejects_wrong_cryptify_token
    • upload_finalize_rejects_size_mismatch
    • upload_finalize_unknown_uuid_returns_404

Each test uses its own temp data_dir under std::env::temp_dir() so they can run in parallel.

Out of scope / follow-ups

The issue mentions MinIO-backed storage behind a trait and >1 MiB chunk coverage. This PR uses on-disk storage (same as production) via a per-test temp dir, which was simpler than abstracting storage. If you want an S3-style backend in tests, that can land as a follow-up. The >1 MiB edge case is partially covered (2 MiB multi-chunk) — a single-chunk ≥ CHUNK_SIZE = 10 MiB is still worth adding once we're happy with this baseline.

Reviewer quickstart

```
git fetch origin && git checkout integration-test-harness && cargo test
```

All 36 tests (27 pre-existing + 9 new) pass locally. cargo clippy --all-targets is clean apart from the pre-existing useless_format warning in src/email.rs:237 noted in Dobby's notes.

Why draft

Keeping as draft so maintainers can decide:

  1. Is email_stub on CryptifyConfig the right shape, or would you prefer a cfg-gated hook / transport trait?
  2. Is the dev-deps rand 0.8 alias acceptable, or would you rather pin pg-core's test feature behind a test-harness cargo feature?
  3. Are you happy with tests living inside src/main.rs (consistent with the existing #[cfg(test)] mod tests), or would you prefer tests/integration.rs? A separate file would mean making build_rocket and a few config bits pub on a new library crate, which is a bigger refactor.

Refs #108.

🤖 Generated with Claude Code

Refactor `rocket()` to extract a `build_rocket(figment, vk)` builder so
tests can inject a `VerifyingKey` (via `pg_core::test::TestSetup`)
without needing a live PKG at startup. Add `email_stub` config flag so
the finalize happy path does not require an SMTP server.

New integration tests in `src/main.rs` (mod `integration`) drive the
full init → chunk(s) → finalize flow against a real Rocket client,
using payloads sealed with pg-core's streaming Sealer. Covers the
happy path (single chunk + multi-chunk), plus cross-handler edge cases
(invalid email, wrong cryptify_token on chunk/finalize, unknown uuid,
size mismatch, content-range misalignment).

Refs #108.
@rubenhensen
Copy link
Copy Markdown
Contributor

/dobby resolve conflicts

@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented Apr 25, 2026

Dobby is here, Dobby sees the review approval! Dobby is so happy that Master rubenhensen approved Dobby's work! Dobby will check the CI checks and, if all is well, Dobby will mark the PR ready for review. Dobby serves!

Resolve conflict around the launch entrypoint by combining both refactors:
- keep `default_figment()` / `build_rocket(figment, vk)` split from this branch
- adopt configurable `chunk_size` (TOML, default 5 MB) from main

Body-size limits (`bytes`, `data-form`, `file`) are now derived from the
extracted CryptifyConfig inside `build_rocket` rather than a constant, since
chunk_size isn't known until the test harness has merged its overrides on
top of `default_figment()`.

All 39 tests (30 pre-existing + 9 integration) pass; clippy is clean apart
from the pre-existing useless_format warning in src/email.rs.
@dobby-coder dobby-coder Bot marked this pull request as ready for review April 25, 2026 11:44
@dobby-coder
Copy link
Copy Markdown
Contributor Author

dobby-coder Bot commented Apr 25, 2026

Resolved merge conflict with main (which had landed configurable chunk_size via TOML in #112) and pushed ceb2e8e. CI is green and the PR is now marked ready for review.

What I changed during the merge

The conflict was around the launch entrypoint where this branch's default_figment() / build_rocket(figment, vk) split met main's new chunk_size-driven body limits. I combined both:

  • default_figment() now just returns rocket::Config::figment() — the body-size limits can't be computed here anymore, because in the test harness default_figment() is called before tests merge their own config on top, so chunk_size isn't known yet.
  • build_rocket() now extracts CryptifyConfig from the figment, derives the limits from config.chunk_size(), merges them in, and only then constructs the Rocket instance. This matches what main did inside #[launch] async fn rocket(), just relocated to the shared builder so production and tests both go through it.

Net effect: production behaviour is unchanged, integration tests still get the same framework-level limits, and chunk_size from config.toml continues to drive bytes / data-form / file limits with a 1 MiB headroom.

Verification

  • cargo build — clean
  • cargo test39 passed, 0 failed (30 pre-existing + 9 integration)
  • cargo clippy --all-targets — clean apart from the pre-existing useless_format warning in src/email.rs (noted in earlier reviews; unrelated)
  • CI: Build (amd64), Build (arm64), and Finalize Docker manifest all passed on ceb2e8e

The three "why draft" questions from the original PR description still stand for reviewer input, but the branch itself is mergeable now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant