internet-latency-collector: select one live probe per cloud region - #4318
Draft
thijsvanemmerik wants to merge 1 commit into
Draft
thijsvanemmerik wants to merge 1 commit into
thijsvanemmerik wants to merge 1 commit into
Conversation
Each region names its own ordered probe list in the node file. Selection takes the first entry that RIPE reports Connected and then sticks to it: an incumbent is replaced only when it stops being Connected or is marked unresponsive, never to re-optimise and never to move back, because recreating a measurement costs the 80 to 100 minutes RIPE takes to dispatch it. A region whose whole list is dead keeps its dead incumbent rather than dropping out, and a probe the node file no longer lists loses the seat. The fallback never widens the search beyond the region's own list: a probe elsewhere would measure the public internet instead of the cloud backbone. Proximity matching, anchor preference and rotation stay on the exchange path. That exchange path also stops indexing a probe's geometry unconditionally. A probe returned with no geometry panicked the measurement cycle at the nearest-probe distance metric, so the distance is now recorded only when both coordinates are present, the same guard the probe fetch in client.go applies.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of measuring latency between AWS regions with the internet latency collector: a node file
switches it into a second mode, the region mode, that measures those regions instead of DoubleZero
exchange locations.
A region with one probe goes dark when that probe disconnects, and the existing recovery fetches
any nearby probe, which for an AWS region could be a home broadband line measuring the public
internet rather than the cloud backbone. This lets the node file list several probe IDs per region,
asks RIPE once per cycle which of them are Connected, and gives each region the first live one from
its own list and nowhere else.
The choice is sticky. A region keeps the probe it already uses while that probe is Connected and not
marked unresponsive, read back from the measurement state with the newest measurement winning. A
region whose candidates are all dead keeps the dead one rather than being rewired to a stranger, and
if the liveness call fails nothing rotates. In the region mode the proximity search and the
nearby-probe fallback do not run at all.
Nothing changes in production: the region mode runs only with a node file, which only a separate
deployment that does not exist yet supplies. One fix does land on the existing path: reading a probe's coordinates now checks the list has two entries first, where a
probe returned without coordinates used to panic with an index out of range.
Test:
go test ./controlplane/internet-latency-collector/internal/ripeatlas/...