Make Sparoid::Instance public-IP resolution thread-safe and self-healing#25
Merged
Conversation
oskgu360
force-pushed
the
fix-instance-public-ip-resolution
branch
5 times, most recently
from
June 29, 2026 12:30
653ad44 to
95a790f
Compare
Sparoid::Instance memoized with `@public_ips ||= super`, which is neither thread-safe nor empty-aware: `[]` is truthy, so a transient empty resolution (icanhazip down or empty body) was pinned for the process lifetime and every later knock then sent zero SPA packets. Under a shared instance hit by a thread pool, the first wave of knocks also resolved concurrently and one slow/empty response clobbered a good value via last-writer-wins. Move the caching into Instance#cached_public_ips: resolve behind a per-instance mutex with a double-check (kills the thundering herd and the clobber) and cache only a non-empty result, so a transient empty resolution self-heals on the next knock instead of pinning the instance. Also raise when generate_messages would otherwise produce no messages: an empty public-IP resolution with no IPv6 fallback used to let auth report success while sending zero packets. It now raises Sparoid::PublicIPError (a sibling of ResolvError under Error) so the caller can tell 'I cannot resolve my own public IP' apart from 'the target host is unreachable', instead of a silent no-op.
oskgu360
force-pushed
the
fix-instance-public-ip-resolution
branch
from
June 29, 2026 12:35
95a790f to
2c1cea6
Compare
oskgu360
marked this pull request as ready for review
June 29, 2026 12:41
There was a problem hiding this comment.
Pull request overview
This PR hardens Sparoid::Instance public-IP resolution by making it thread-safe, avoiding permanently caching empty resolutions, and by failing fast when an auth attempt would otherwise send zero SPA packets.
Changes:
- Add
Sparoid::PublicIPErrorand raise it when no public IP can be resolved for message generation. - Implement per-instance, mutex-guarded, non-empty memoization via
Instance#cached_public_ips. - Add WebMock-backed tests for retry/self-healing behavior and for the new error path; add
webmockto dev deps.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| lib/sparoid.rb | Adds PublicIPError, raises on empty IP resolution, and introduces mutex-protected per-instance caching of public IPs. |
| test/test_helper.rb | Enables WebMock for the suite (but currently allows real network connections). |
| test/sparoid_test.rb | Adds WebMock stubs and tests covering non-pinning empty results and raising on zero-message auth. |
| Gemfile | Adds webmock for development/test usage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Author
|
Ran ssh-monitor fix with this patch locally (just verified it were able to ssh). Will rescue PublicIPError to handle that gracefully in case of this happening again. If we decide to merge this that is |
walro
reviewed
Jun 29, 2026
Drop WebMock.allow_net_connect! and stub icanhazip in test_it_resolves_public_ip so the suite no longer depends on a working internet connection or icanhazip being up (which would make CI flaky and let mistyped stubs fall through to the network).
@public_ips is now always an array, so the empty/cached check is a plain .any? instead of &.any?.
walro
approved these changes
Jun 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Sparoid::Instancememoized its public IP with@public_ips ||= super, which has two defects:[]is truthy in Ruby, so once@public_ipsis[]the||=short-circuits and never re-resolves. A single transient empty resolution (icanhazip down or returning an empty body) poisons the instance for its whole life — every later knock then builds zero messages andauthsends zero SPA packets.||=is read-modify-write. When oneInstanceis shared across a thread pool, the first wave of knocks all resolve concurrently (a herd of icanhazip requests) and one slow/empty response clobbers a good value via last-writer-wins — then defect Include gem name in warnings #1 pins it.This was caught in production (ssh-monitor worker.26, 2026-06-26): a dyno sent zero-packet knocks for its whole lifetime and alarmed every host as SSH-down. ssh-monitor currently carries an app-side monkey-patch for this; this PR moves the proper fix into the gem.
Fix
Instance#cached_public_ips(calling the inherited rawpublic_ipsfetch directly — nosuper, nopublic_ipsoverride).generate_messagesnow raises when an empty public-IP resolution and no IPv6 fallback would otherwise produce no messages, instead of lettingauthreport success having sent nothing.Sparoid::PublicIPError— a sibling ofResolvErrorunderError(so existingrescue Sparoid::Errorstill catches it), letting callers distinguish "I can't resolve my own public IP" (a client-side failure affecting every host) from "the target host is unreachable." Notably it is not aResolvError, so ssh-monitor'srescue Sparoid::ResolvError"machine deleted" no-op won't silently swallow it.Tests
test_instance_retries_and_recovers_instead_of_pinning_empty— empty resolve is not memoized; recovers once icanhazip responds again (webmock).test_auth_raises_when_no_public_ip_resolved—authraisesSparoid::PublicIPErrorrather than silently sending zero packets.Adds
webmockas a dev dependency.Note
Version bump + CHANGELOG and the ssh-monitor cleanup are intentionally left to separate follow-up PRs. The follow-up can rescue
Sparoid::PublicIPErrorspecifically so an icanhazip outage doesn't mark every host as SSH-down.