Repository navigation
Implement a HttpClient that can do multiple downloads in parallel - #3464
Merged
Merged
Conversation
Databean
changed the title
jemoreira
approved these changes
Oct 30, 2026
Databean
force-pushed
the
curl_parallel
branch
from
October 30, 2026 21:41
8c6dc01 to
eaeddc6
Compare
Databean
force-pushed
the
curl_parallel
branch
from
October 30, 2026 21:56
eaeddc6 to
9a3cf1f
Compare
Delegate HttpClient implementations are created dynamically up to a limit. Bug: b/579555638
Behavioral difference on my machine, running ``` time bazel run //cuttlefish/package:cvd -c opt -- fetch \ --default_build=git_main/aosp_cf_x86_64_only_phone-trunk_staging-userdebug \ --target_directory=$HOME/dl \ --enable_caching=false ``` Without pool, 5 runs: - 15.24s user 11.90s system 66% cpu 40.926 total - 14.85s user 11.96s system 69% cpu 38.312 total - 15.10s user 12.24s system 69% cpu 39.434 total - 14.94s user 12.08s system 70% cpu 38.171 total - 15.58s user 12.09s system 69% cpu 40.081 total With pool, 5 runs: - 15.31s user 14.47s system 79% cpu 37.281 total - 15.39s user 12.45s system 74% cpu 37.200 total - 14.82s user 12.66s system 76% cpu 35.918 total - 14.98s user 12.98s system 77% cpu 36.279 total - 14.94s user 13.37s system 75% cpu 37.521 total Right now, most of the invocations of `HttpClient` happen synchronously, except for the cvd-host_package.tar.gz download, which is always invoked on a separate thread. Bug: b/579555638
Databean
force-pushed
the
curl_parallel
branch
from
October 30, 2026 22:04
9a3cf1f to
77a7298
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This will allow implementing more parallelism in cvd fetch.
Behavioral difference on my machine, running
Without pool, 5 runs:
With pool, 5 runs:
Right now, most of the invocations of
HttpClienthappen synchronously, except for the cvd-host_package.tar.gz download, which is always invoked on a separate thread.Bug: b/579555638