chore(tests): replace localstack with moto for AWS integration tests#25188
Draft
thomasqueirozb wants to merge 6 commits intomasterfrom
Draft
chore(tests): replace localstack with moto for AWS integration tests#25188thomasqueirozb wants to merge 6 commits intomasterfrom
thomasqueirozb wants to merge 6 commits intomasterfrom
Conversation
Co-authored-by: Thomas <thomasqueirozb@gmail.com>
LocalStack 2026.03.0 introduced a mandatory auth token requirement whose grace period expired on 2026-04-06, breaking all CI runs that used the free/open-source image. Switch to motoserver/moto, which is a fully open-source, no-auth-required AWS mock. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LocalStack 2026.03.0 introduced a mandatory auth token requirement. The grace period for existing users expired on 2026-04-06, after which all CI runs using the free
localstack/localstackimage started failing with authentication errors (see #24988, which had already pinned to a specific digest as a temporary workaround).This PR replaces LocalStack with motoserver/moto — a fully open-source, no-auth-required AWS mock — across the AWS integration test infrastructure.
Changes
tests/integration/aws/config/compose.yaml: Replacemock-localstack(LocalStack image) withmock-aws(motoserver/moto). Remove theSERVICES=env var (moto needs no service selection).tests/integration/aws/config/test.yaml: Update all endpoint env vars fromhttp://mock-localstack:4566tohttp://mock-aws:5000. Rename the matrix key fromlocalstack_versiontomoto_version. RemoveELASTICSEARCH_ADDRESS(no longer needed).src/sinks/aws_kinesis/firehose/integration_tests.rs: Rewrite to use an S3 delivery destination instead of Elasticsearch. Moto's Firehose implementation delivers records to its internal S3 backend for S3/ExtendedS3 destinations; the Elasticsearch destination is present in moto but ignores record data. The test now creates an S3 bucket, points Firehose at it, sends records, then reads them back via the S3 API.Cargo.toml: Remove the unusedaws-sdk-elasticsearchdependency (no source files reference it after the Firehose test rewrite). Update theaws-kinesis-firehose-integration-testsfeature to depend onsinks-aws_s3instead.Limitations / Known Issues
CloudWatch Metrics tests will still fail
aws-sdk-cloudwatch≥ 1.100.0 switched from the legacy AWS Query Protocol to smithy-rpc-v2 CBOR, sendingPutMetricDatarequests to:Moto's CloudWatch Flask app only registers a route for
/(the old query protocol). The CBOR path returns 404. Notably,put_metric_datais marked as implemented in moto's IMPLEMENTATION_COVERAGE.md — the operation logic exists, only the CBOR transport layer is missing.Possible fixes (not done here, low priority):
aws-sdk-cloudwatchto=1.99.0(last version before CBOR; compatible with all current smithy runtime versions inCargo.lock)cbor2and forwarding to the existing backendcompose.yamlVector configuration
N/A — test infrastructure change only.
How did you test this PR?
Ran the AWS integration tests locally with
cargo vdev int start aws && cargo vdev int test aws. Firehose and Kinesis Streams tests pass. CloudWatch Metrics tests fail due to the CBOR limitation described above.Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.References