Skip to content

YARN-11813. Fix the fallback ordering between cgroup v2 and v1. #7631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 25, 2025
+25 −6
Merged

YARN-11813. Fix the fallback ordering between cgroup v2 and v1. #7631

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a5370e8
YARN-11813. Fix the fallback ordering between cgroup v2 and v1.
brumi1024 Apr 17, 2025
ddd9f01
Update based on comments.
brumi1024 Apr 22, 2025
ceb7be7
Fix linting issues.
brumi1024 Apr 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions ...hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsV2HandlerImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,10 +95,22 @@ protected List<CGroupController> getCGroupControllers() {
}

@Override
protected Map<String, Set<String>> parsePreConfiguredMountPath() throws IOException {
protected Map<String, Set<String>> parsePreConfiguredMountPath() {
Map<String, Set<String>> controllerMappings = new HashMap<>();
controllerMappings.put(this.cGroupsMountConfig.getV2MountPath(),
readControllersFile(this.cGroupsMountConfig.getV2MountPath()));
try {
controllerMappings.put(this.cGroupsMountConfig.getV2MountPath(),
readControllersFile(this.cGroupsMountConfig.getV2MountPath()));
} catch (IOException e) {
// Failing to read the cgroup.controllers file in the preconfigured might mean
// that the node is not using cgroup v2, or no cgroup v2 hierarchy is mounted
// under the specified path. If the node is using v1 we will fall back to cgroup v1
// in ResourceHandlerModule.initializeCGroupHandlers. If the cgroup v2 hierarchy is
// not mounted and no cgroup v1 hierarchy is mounted, we will fail to start the NM.
LOG.info("Failed to read the cgroup controllers file in the preconfigured directory: {}. " +
"The cgroup v2 hierarchy may not be mounted under the specified path, or the node" +
" might be using cgroup v1.", this.cGroupsMountConfig.getV2MountPath());
LOG.debug("Exception while reading the cgroup.controllers file: ", e);
}
return controllerMappings;
}

Expand Down
13 changes: 10 additions & 3 deletions ...adoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,17 @@ public class ResourceHandlerModule {
cGroupsCpuResourceHandler;

private static void initializeCGroupHandlers(Configuration conf,
CGroupsHandler.CGroupController controller) throws ResourceHandlerException {
initializeCGroupV1Handler(conf);
if (cgroupsV2Enabled && !isMountedInCGroupsV1(controller)) {
CGroupsHandler.CGroupController controller)
throws ResourceHandlerException {
if (cgroupsV2Enabled) {
initializeCGroupV2Handler(conf);
if (!isMountedInCGroupsV2(controller)) {
LOG.info("Cgroup v2 is enabled but {} is not mounted in cgroups v2, falling back to v1",
controller);
initializeCGroupV1Handler(conf);
}
} else {
initializeCGroupV1Handler(conf);
}
}

Expand Down