Skip to content

[v0.21.x-branch] Backport #10771: routerrpc: isolate LSP route fee probes#10834

Merged
ziggie1984 merged 1 commit into
v0.21.x-branchfrom
backport-10771-to-v0.21.x-branch
May 22, 2026
Merged

[v0.21.x-branch] Backport #10771: routerrpc: isolate LSP route fee probes#10834
ziggie1984 merged 1 commit into
v0.21.x-branchfrom
backport-10771-to-v0.21.x-branch

Conversation

@github-actions
Copy link
Copy Markdown

@github-actions github-actions Bot commented May 22, 2026

Backport of #10771

The last commit which added the release notes where already added in commit: df04aaf so we drop the last commit here.


Summary

  • Use a fresh payment hash for each EstimateRouteFee LSP probe so later probes cannot reuse payment-level state from earlier probes.
  • Add deterministic unit coverage that stubs probe dispatch and verifies each LSP probe has a distinct payment hash and the expected final CLTV delta.
  • Add a release note for the multi-LSP EstimateRouteFee fix.

Fix #10677

Bug

CI flaked in PR #10689 on the bitcoind-etcd itest job:

https://github.com/lightningnetwork/lnd/actions/runs/24841352445/job/72716012767?pr=10689

Failed test:

TestLightningNetworkDaemon/tranche03/141-of-347/bitcoind/estimate_route_fee/probe_based_estimate,_multiple_different_public_LSPs

Failure:

Error:       Not equal:
             expected: 944
             actual  : 844
Messages:    cltv delta

The test creates three public LSP hints for a private destination:

  • Bob: high-level expected final LSP CLTV delta 100
  • Eve: highest-fee LSP, final LSP CLTV delta 200
  • Dave: lower-fee LSP, final LSP CLTV delta 120

The returned fee was Eve's fee, but the returned timelock was calculated with Bob's CLTV delta.

Root Cause

probePaymentRequest generated one random payment hash for the overall invoice probe request, then reused that same hash for every per-LSP probe.

The payment lifecycle keys payment state by payment hash. If Bob is probed first, the payment record is initialized with Bob's FinalCltvDelta=100. When Eve is probed later with the same payment hash, the router treats it as another attempt for the same payment and can reuse payment-level state from the first probe. The route can then carry Eve's fee but Bob's CLTV delta.

This is order-dependent because LSPs are iterated from a Go map. If Eve is probed first, the test passes. If Bob is probed first, Eve can inherit Bob's CLTV delta and the test fails.

Log Evidence

I downloaded the itest artifact logs-itest-bitcoind-etcd.zip from the failed run. Relevant log file:

itest/.logs-tranche3/22-estimate_route_fee-Alice-0223edc2.log

Passing two-LSP case: Eve is probed first, uses expiry/cltv 864, and the final result returns timelock: 944.

1884: 2026-04-23 14:53:05.208 [INF] RRPC router_server.go:615: Probing LSP with destination: 024b44330ad86737ecaa6f8a8bf1b10b7ab0d14d94631934bc02e9d3d4716d8880
1891: 2026-04-23 14:53:05.216 [DBG] CRTR payment_lifecycle.go:734: Attempt 6 for payment 2f2c715fe2abb8fed05b39cb5ce57e06ccb694655a3191928fc6a11dfd051db2 successfully sent to switch, route: 630020162846720 (101001201 mSAT) -> 630020162912256 (101000100 mSAT), cltv 864
1895: 2026-04-23 14:53:05.216 [DBG] PEER brontide.go:2809: Peer(03b85df66a282b9bcc9a636fb7a7948f2b5377072d55cfc8e1b15ba18851bc920d): Sending UpdateAddHTLC(chan_id=075f4a0a9a0e5bdfc7371304ba2c5a22342aea5c900110a4e3481ab69c704c62, id=5, amt=101001201 mSAT, expiry=864, hash=2f2c715fe2abb8fed05b39cb5ce57e06ccb694655a3191928fc6a11dfd051db2, blinding_point=, custom_records=map[106823:[0]]) to 03b85df66a282b9bcc9a636fb7a7948f2b5377072d55cfc8e1b15ba18851bc920d@127.0.0.1:11145
1920: 2026-04-23 14:53:05.575 [INF] RRPC router_server.go:677: Probe to LSP 024b44330ad86737ecaa6f8a8bf1b10b7ab0d14d94631934bc02e9d3d4716d8880 succeeded with fee: 1001201 msat
1965: 2026-04-23 14:53:05.991 [INF] RRPC router_server.go:711: Returning worst-case route via LSP 024b44330ad86737ecaa6f8a8bf1b10b7ab0d14d94631934bc02e9d3d4716d8880 with fee: 1001201 msat, timelock: 944

Failing three-LSP case: Bob is probed first with payment hash a539... and cltv 764. Eve is probed next with the same payment hash, logs Resuming HTLC attempt, still uses cltv 764, succeeds with Eve's fee, and returns timelock: 844.

1992: 2026-04-23 14:53:06.034 [INF] RRPC router_server.go:615: Probing LSP with destination: 031fea722979dc497b8f5cf6e3c240bfadcee7c1084636f6e2941676935ece12a9
1999: 2026-04-23 14:53:06.043 [DBG] CRTR payment_lifecycle.go:734: Attempt 8 for payment a5397f710a5edc3686412161ac38c38779b1f9ad4c62c15648feb883f6ea7743 successfully sent to switch, route: 630020162846720 (100002200 mSAT) -> 630020162977792 (100001100 mSAT), cltv 764
2003: 2026-04-23 14:53:06.043 [DBG] PEER brontide.go:2809: Peer(03b85df66a282b9bcc9a636fb7a7948f2b5377072d55cfc8e1b15ba18851bc920d): Sending UpdateAddHTLC(chan_id=075f4a0a9a0e5bdfc7371304ba2c5a22342aea5c900110a4e3481ab69c704c62, id=7, amt=100002200 mSAT, expiry=764, hash=a5397f710a5edc3686412161ac38c38779b1f9ad4c62c15648feb883f6ea7743, blinding_point=, custom_records=map[106823:[0]]) to 03b85df66a282b9bcc9a636fb7a7948f2b5377072d55cfc8e1b15ba18851bc920d@127.0.0.1:11145
2028: 2026-04-23 14:53:06.428 [INF] RRPC router_server.go:677: Probe to LSP 031fea722979dc497b8f5cf6e3c240bfadcee7c1084636f6e2941676935ece12a9 succeeded with fee: 2200 msat
2029: 2026-04-23 14:53:06.428 [INF] RRPC router_server.go:615: Probing LSP with destination: 024b44330ad86737ecaa6f8a8bf1b10b7ab0d14d94631934bc02e9d3d4716d8880
2031: 2026-04-23 14:53:06.432 [DBG] CRTR payment_lifecycle.go:1180: Payment a5397f710a5edc3686412161ac38c38779b1f9ad4c62c15648feb883f6ea7743: status=Initiated, active_shards=0, rem_value=101000100 mSAT, fee_limit=100000000000 mSAT
2037: 2026-04-23 14:53:06.439 [INF] CRTR payment_lifecycle.go:1153: Resuming HTLC attempt 9 for payment a5397f710a5edc3686412161ac38c38779b1f9ad4c62c15648feb883f6ea7743
2040: 2026-04-23 14:53:06.441 [DBG] CRTR payment_lifecycle.go:734: Attempt 9 for payment a5397f710a5edc3686412161ac38c38779b1f9ad4c62c15648feb883f6ea7743 successfully sent to switch, route: 630020162846720 (101001201 mSAT) -> 630020162977792 (101000100 mSAT), cltv 764
2043: 2026-04-23 14:53:06.441 [DBG] PEER brontide.go:2809: Peer(03b85df66a282b9bcc9a636fb7a7948f2b5377072d55cfc8e1b15ba18851bc920d): Sending UpdateAddHTLC(chan_id=075f4a0a9a0e5bdfc7371304ba2c5a22342aea5c900110a4e3481ab69c704c62, id=8, amt=101001201 mSAT, expiry=764, hash=a5397f710a5edc3686412161ac38c38779b1f9ad4c62c15648feb883f6ea7743, blinding_point=, custom_records=map[106823:[0]]) to 03b85df66a282b9bcc9a636fb7a7948f2b5377072d55cfc8e1b15ba18851bc920d@127.0.0.1:11145
2076: 2026-04-23 14:53:06.816 [INF] RRPC router_server.go:677: Probe to LSP 024b44330ad86737ecaa6f8a8bf1b10b7ab0d14d94631934bc02e9d3d4716d8880 succeeded with fee: 1001201 msat
2124: 2026-04-23 14:53:07.067 [INF] RRPC router_server.go:711: Returning worst-case route via LSP 024b44330ad86737ecaa6f8a8bf1b10b7ab0d14d94631934bc02e9d3d4716d8880 with fee: 1001201 msat, timelock: 844

The 100 delta between expected 944 and actual 844 matches Bob's LSP CLTV delta being used where Eve's 200 should have been used.

Fix

Generate a fresh random payment hash for each LSP probe iteration. This makes each LSP probe an independent payment from the payment lifecycle's perspective, while keeping the existing single-hash behavior for the non-LSP probe path.

Testing

  • GOWORK=off go test ./lnrpc/routerrpc

@github-actions github-actions Bot added this to the v0.21.0 milestone May 22, 2026
@github-actions
Copy link
Copy Markdown
Author

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-10771-to-v0.21.x-branch
git worktree add --checkout .worktree/backport-10771-to-v0.21.x-branch backport-10771-to-v0.21.x-branch
cd .worktree/backport-10771-to-v0.21.x-branch
git reset --hard HEAD^
git cherry-pick -x 8eab1edd69887e4935b3e06eccf558902b29ffdd
git push --force-with-lease

@ziggie1984 ziggie1984 force-pushed the backport-10771-to-v0.21.x-branch branch from 2ca6bf3 to c709ec5 Compare May 22, 2026 12:31
@ziggie1984 ziggie1984 marked this pull request as ready for review May 22, 2026 12:32
@ziggie1984 ziggie1984 merged commit a9d8ae2 into v0.21.x-branch May 22, 2026
34 of 37 checks passed
@github-actions github-actions Bot added the severity-high Requires knowledgeable engineer review label May 22, 2026
@github-actions
Copy link
Copy Markdown
Author

PR Severity: HIGH

Classified from changed files | 1 non-test file | 36 lines changed

High (1 file)
  • lnrpc/routerrpc/router_server.go - Router RPC server under lnrpc/* (RPC/API definitions)
Excluded from classification (1 file)
  • lnrpc/routerrpc/router_server_test.go - test file (excluded)

Analysis

This PR modifies lnrpc/routerrpc/router_server.go under the lnrpc/* package category (RPC/API definitions), placing it at HIGH severity requiring a knowledgeable engineer review.

Scope: 1 non-test file, 36 lines changed -- no bump triggered (thresholds: more than 20 non-test files or more than 500 non-test lines).


To override, add a severity-override-{critical,high,medium,low} label.
<!-- pr-severity-bot -->

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog severity-high Requires knowledgeable engineer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants