Skip to content

YARN-11971: Add validation to block adding queue capacity in partition view if the queue does not have the accessible label configured - #8641

Open
Hean-Chhinling wants to merge 3 commits into
apache:trunkfrom
Hean-Chhinling:trunk_YARN-11971
Open

YARN-11971: Add validation to block adding queue capacity in partition view if the queue does not have the accessible label configured#8641
Hean-Chhinling wants to merge 3 commits into
apache:trunkfrom
Hean-Chhinling:trunk_YARN-11971

Conversation

@Hean-Chhinling

@Hean-Chhinling Hean-Chhinling commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Description of PR

Queue capacity could be set in partition view even though the queue does not have access to that label. Leaving the only queue's label capacity in /scheduler-conf endpoint. E.g.

<property>
<name>yarn.scheduler.capacity.root.default.accessible-node-labels.label3.capacity</name>
<value>100</value>
</property>

Resulting into a bug when the label is deleted and created a new queue. Detail in the jira ticket description.

Therefore, a new validation is needed to make sure the queue has access to the label before allowing to set capacity in the partition view. Vice versa, the accessible label could not be removed if the queue has capacity in that partition. Both have to be changed at the same time.

How was this patch tested?

Tested locally plus unit tests and work as expected.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 49s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 43m 58s trunk passed
+1 💚 shadedclient 73m 16s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 30s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 shadedclient 28m 34s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 45s The patch does not generate ASF License warnings.
106m 47s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8641/1/artifact/out/Dockerfile
GITHUB PR #8641
Optional Tests dupname asflicense shadedclient codespell detsecrets
uname Linux 5cd753a244de 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 48c7d58
Max. process+thread count 640 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8641/1/console
versions git=2.43.0 maven=3.9.15
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 36s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 💚 mvninstall 44m 48s trunk passed
+1 💚 shadedclient 75m 25s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 29s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 shadedclient 29m 16s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 44s The patch does not generate ASF License warnings.
109m 26s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8641/2/artifact/out/Dockerfile
GITHUB PR #8641
Optional Tests dupname asflicense shadedclient codespell detsecrets
uname Linux d7dfc04ee9c4 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 853b239
Max. process+thread count 635 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8641/2/console
versions git=2.43.0 maven=3.9.15
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@K0K0V0K

K0K0V0K commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Thanks @Hean-Chhinling for taking care of this issue!

If i understand correctly with the mutable conf api the users can creates capacity configs what will be invalid.
In that case, this validation should happen in the CapacitySchedulerConfigValidator at the first place.
This UI validation also can be useful.

May i ask you to attach some screenshot to the jira ticket how the manual test happened?

@Hean-Chhinling

Copy link
Copy Markdown
Contributor Author

Thanks @Hean-Chhinling for taking care of this issue!

If i understand correctly with the mutable conf api the users can creates capacity configs what will be invalid. In that case, this validation should happen in the CapacitySchedulerConfigValidator at the first place. This UI validation also can be useful.

May i ask you to attach some screenshot to the jira ticket how the manual test happened?

You are right @K0K0V0K!
I should have stopped the misconfiguration from the backend not just at the UI.
I will update the code in my new commit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants