Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Critical Bug in Unfreeze Propagation #6631

Open
umarsync opened this issue Jan 26, 2025 · 2 comments
Open

Critical Bug in Unfreeze Propagation #6631

umarsync opened this issue Jan 26, 2025 · 2 comments

Comments

@umarsync
Copy link

Describe the bug

In the cgroup_propagate_frozen function, during unfreezing (when frozen is false), the code subtracts desc (which increments for each unfrozen parent) from each ancestor's nr_frozen_descendants. This results in over-subtraction, leading to negative counts and incorrect frozen states.

kernel/cgroup/freezer.c

Example Scenario:

  • Hierarchy: A -> B -> C
  • All cgroups (A, B, C) are frozen.
  • Unfreeze C:
  • Current Code: Subtracts desc=1 from B's count (correctly setting it to 0). Then subtracts desc=2 from A's count, reducing it from 2 to 0. If A has another frozen child D, this incorrectly sets A's count to 0 instead of 1 (since D is still frozen).

Additional Considerations

  • Negative Count Prevention: Added WARN_ON_ONCE to catch underflows, though proper decrement by 1 should prevent negatives.
  • Propagation Continuation: The loop continues upwards until all affected ancestors are updated.

The identified bug leads to incorrect frozen state propagation, particularly in hierarchical cgroups. The proposed fix corrects the decrement logic during unfreezing, ensuring accurate nr_frozen_descendants counts and proper state transitions. This issue warrants a kernel patch to prevent system instability from incorrect cgroup freezing.

Steps to reproduce the behaviour

Test Case: Incorrect unfreezing of parent cgroup when one of multiple frozen descendants is unfrozen.

  1. Setup Hierarchy:
# Create cgroups
mkdir /sys/fs/cgroup/A
mkdir /sys/fs/cgroup/A/B
mkdir /sys/fs/cgroup/A/D
mkdir /sys/fs/cgroup/A/B/C
  1. Freeze Leaf Cgroups:
# Freeze C and D (A and B will auto-freeze via propagation)
echo 1 > /sys/fs/cgroup/A/B/C/cgroup.freeze
echo 1 > /sys/fs/cgroup/A/D/cgroup.freeze
  1. Verify All Cgroups Are Frozen:
cat /sys/fs/cgroup/A/cgroup.freeze # Should show 1 (frozen)
cat /sys/fs/cgroup/A/B/cgroup.freeze # Should show 1 (frozen)
cat /sys/fs/cgroup/A/B/C/cgroup.freeze # Should show 1 (frozen)
cat /sys/fs/cgroup/A/D/cgroup.freeze # Should show 1 (frozen)
  1. Unfreeze Cgroup C:
echo 0 > /sys/fs/cgroup/A/B/C/cgroup.freeze
  1. Observe Incorrect Behavior:
cat /sys/fs/cgroup/A/cgroup.freeze # ❌ Shows 0 (unfrozen) **BUG**
cat /sys/fs/cgroup/A/D/cgroup.freeze # Still shows 1 (frozen)

Expected Behavior:

  • A should remain frozen because its descendant D is still frozen.
  • B should unfreeze (correctly) because its only descendant C was unfrozen.

Actual Behavior (with Bug):

  • A incorrectly unfreezes due to over-decrement in nr_frozen_descendants.

Debugging Tips:

  1. Check kernel logs for warnings like:
WARN_ON_ONCE(cgrp->freezer.nr_frozen_descendants < 0)
  1. Use ftrace to monitor cgroup_propagate_frozen calls:
echo 1 > /sys/kernel/debug/tracing/events/cgroup/enable
cat /sys/kernel/debug/tracing/trace_pipe

Device (s)

Other

System

rpi-6.6.y

Logs

No response

Additional context

No response

@popcornmix
Copy link
Collaborator

kernel/cgroup/freezer.c is unchanged from upstream.
I think you'll have better luck reporting this issue upstream - it is unlikely to be a Pi specific issue,
and there may be developers of this actual code who can comment.

Your detailed report should be helpful to the maintainers.

See: https://docs.kernel.org/admin-guide/reporting-issues.html

@pelwell
Copy link
Contributor

pelwell commented Jan 30, 2025

Yes - see dialog at #6632.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants