HBASE-30099 Avoid NPE when GetBootstrapNodes RPC arrives during startup - #8483
Conversation
|
Hi maintainers, the three GitHub Actions workflows for this fork PR are awaiting approval. This PR does not modify any workflow files. Could someone with write access approve them when convenient? Thanks. |
|
Hi @liuxiaocs7, sorry for the direct ping. Could you please help approve the fork workflows for this PR, or suggest an appropriate reviewer, when convenient? The original reproducer has been verified successfully. Thank you! |
|
Hi @wchevreuil and @ndimiduk, could you please take a look at this PR when convenient? This patch fixes the RegionServer startup race reported in HBASE-30099. During RegionServer initialization, The proposed fix makes the RPC return an empty bootstrap-node list during this short initialization window. Once @ndimiduk, since you reported HBASE-30099, it would also be helpful if you could confirm whether returning an empty list during startup matches the expected behavior. The PR is currently mergeable and has not received a review yet. If the forked GitHub Actions workflows still require approval, could a committer please approve them as well? Thanks! |
|
Hi @Sigma-Ma, apologies for the delay on this PR. I don't see an option to enable the Github Action, maybe it was idle for too long. Could you do a force push? Hopefully that will help. |
627dbaf to
8409f92
Compare
Thanks @petersomogyi. I applied the suggestion, amended the commit. Could you please check whether the GitHub Actions approval option is available now? |
8409f92 to
e1666af
Compare
|
Hi @petersomogyi. The remaining failure looks unrelated to this patch. TestSyncReplicationStandbyKillMaster failed in the large-wave-3 shard because the RegionServer still reported the peer as STANDBY after the transition, and the retries then ran into stale mini-cluster state. The General Check, Compile Check, and the other four unit-test shards all passed. Could you please rerun the failed job when you have a chance? |
|
A rerun of wave3 was successful. |
What changes were proposed in this pull request?
Make
HRegionServer#getBootstrapNodesreturn an empty iterator whenbootstrapNodeManagerhas not been initialized yet.Declare
bootstrapNodeManageras volatile to ensure safe publication between the RegionServer startup thread and RPC handler threads.Add a regression test covering calls to
getBootstrapNodesbeforebootstrapNodeManagerinitialization.Why are the changes needed?
The RPC server can start accepting connections before
preRegistrationInitialization()assignsbootstrapNodeManager.If a
GetBootstrapNodesRPC arrives during this startup window,HRegionServer#getBootstrapNodesdereferences a nullbootstrapNodeManagerand throws aNullPointerException.Returning an empty iterator during this window avoids the startup race and allows callers to continue bootstrap-node discovery normally.
How was this patch tested?
mvn -pl hbase-server -am \ -Dtest=TestBootstrapNodeManager \ -Dsurefire.failIfNoSpecifiedTests=false \ -DskipITs \ test