[Bug] BE division-by-zero error occurs when calculating inode_percentage

[uchenily]

Search before asking

• I had searched in the issues and found no similar issues.

Version

4.1 7126cf6 cloud version

What's Wrong?

In some environment, the inode statistics f_files/f_ffree are 0, which causes a BE division-by-zero error.

inode_total = stat.f_files = 0
inode_free = stat.f_ffree = 0

Thread 185 "run_background_" received signal SIGFPE, Arithmetic exception.
0x000055556bce32e8 in doris::io::disk_used_percentage (path=..., percent=0x7fff109ead50) at /home/chen/Programming/doris-4.1/be/src/io/cache/block_file_cache.cpp:1822
1822	    int inode_percentage = cast_set<int>(inode_free * 100 / inode_total);
(gdb) bt
#0  0x000055556bce32e8 in doris::io::disk_used_percentage (path=..., percent=0x7fff109ead50) at /home/chen/Programming/doris-4.1/be/src/io/cache/block_file_cache.cpp:1822
#1  0x000055556bce422a in doris::io::BlockFileCache::check_disk_resource_limit (this=0x7fff151c6000) at /home/chen/Programming/doris-4.1/be/src/io/cache/block_file_cache.cpp:1901
#2  0x000055556bce564d in doris::io::BlockFileCache::run_background_monitor (this=0x7fff151c6000) at /home/chen/Programming/doris-4.1/be/src/io/cache/block_file_cache.cpp:1998
#3  0x000055556bd313c3 in std::__invoke_impl<void, void (doris::io::BlockFileCache::*)(), doris::io::BlockFileCache*> (
    __f=@0x7fff14da6ff0: (void (doris::io::BlockFileCache::*)(doris::io::BlockFileCache * const)) 0x55556bce54b4 <doris::io::BlockFileCache::run_background_monitor()>, __t=@0x7fff14da6fe8: 0x7fff151c6000)
    at /home/chen/Programming/doris-4.1/ldb_toolchain/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:76
#4  0x000055556bd31228 in std::__invoke<void (doris::io::BlockFileCache::*)(), doris::io::BlockFileCache*> (
    __fn=@0x7fff14da6ff0: (void (doris::io::BlockFileCache::*)(doris::io::BlockFileCache * const)) 0x55556bce54b4 <doris::io::BlockFileCache::run_background_monitor()>)
    at /home/chen/Programming/doris-4.1/ldb_toolchain/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:98
#5  0x000055556bd30a99 in std::thread::_Invoker<std::tuple<void (doris::io::BlockFileCache::*)(), doris::io::BlockFileCache*> >::_M_invoke<0ul, 1ul> (this=0x7fff14da6fe8)
    at /home/chen/Programming/doris-4.1/ldb_toolchain/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:303
#6  0x000055556bd3078c in std::thread::_Invoker<std::tuple<void (doris::io::BlockFileCache::*)(), doris::io::BlockFileCache*> >::operator() (this=0x7fff14da6fe8)
    at /home/chen/Programming/doris-4.1/ldb_toolchain/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:310
#7  0x000055556bd30222 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (doris::io::BlockFileCache::*)(), doris::io::BlockFileCache*> > >::_M_run (this=0x7fff14da6fe0)
    at /home/chen/Programming/doris-4.1/ldb_toolchain/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:255
#8  0x000055557c6bc014 in execute_native_thread_routine ()
#9  0x00007ffff6962d19 in start_thread () from /lib64/libc.so.6
#10 0x00007ffff69e664c in __clone3 () from /lib64/libc.so.6
(gdb) p stat
$5 = {f_type = 2435016766, f_bsize = 4096, f_blocks = 124348672, f_bfree = 83618897, f_bavail = 83530293, f_files = 0, f_ffree = 0, f_fsid = {__val = {-1491911987, -353343286}}, f_namelen = 255, f_frsize = 4096, 
  f_flags = 4128, f_spare = {0, 0, 0, 0}}
(gdb) p inode_free
$6 = 0
(gdb) p inode_total
$7 = 0

file system infos:

$ df -T /home/chen/Programming/doris-4.1/output/be/file_cache
文件系统       类型     1K的块      已用      可用 已用% 挂载点
/dev/nvme0n1p3 btrfs 497394688 163018220 334023188   33% /home

What You Expected?

not coredump

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[zclllyybb]

Breakwater-GitHub-Analysis-Slot: slot_da6fafffe61f
This content is generated by AI for reference only.

Initial analysis for #65545:

The stack trace points to a real BE bug in the file-cache disk monitor path, not only an environment/configuration issue. In the reported commit 7126cf65d96ebc43fce0906f51e92c1a2ccf24a6 (4.1.3-rc02), be/src/io/cache/block_file_cache.cpp::disk_used_percentage() reads stat.f_ffree and stat.f_files, then computes:

int inode_percentage = cast_set<int>(inode_free * 100 / inode_total);

There is no guard for inode_total == 0. The issue's GDB output shows stat.f_files = 0 and stat.f_ffree = 0, so the SIGFPE at that line is expected from the current code. I also checked refreshed upstream refs: the same unguarded expression is still present on upstream/branch-4.1 (598749afd1d) and upstream/master (8b52996cd9a).

Impact:

• The crash occurs in the background file-cache monitor: BlockFileCache::run_background_monitor() -> check_disk_resource_limit() -> disk_used_percentage().
• It can happen when the cache path is on a filesystem that reports block capacity but does not expose inode accounting through statfs() (f_files/f_ffree == 0).
• statfs() itself succeeds in this case, so the existing ret != 0 error handling is bypassed.

Suggested fix:

• Treat stat.f_files == 0 as "inode usage unavailable" in disk_used_percentage() instead of dividing by it.
• A practical behavior is to keep the existing disk-capacity calculation and set inode used percent to 0 or otherwise skip inode-pressure decisions when total inode count is unavailable. This keeps space-based protection active and avoids falsely entering inode-pressure mode on filesystems that do not report inodes.
• This matches nearby file-cache behavior: FSFileCacheStorage::estimate_file_count_from_inode() already checks vfs.f_files == 0 and skips inode estimation instead of using it.
• If the helper is being hardened more generally, it is also worth guarding nonroot_total == 0, although the evidence in this issue specifically proves the inode denominator problem.

Suggested tests:

• Add a BE unit test for disk_used_percentage() or for a factored statfs-result helper with f_files = 0 and f_ffree = 0, asserting no SIGFPE and a bounded returned percentage pair.
• The existing BlockFileCache::disk_used_percentage:1 sync point runs after the calculation, so it cannot reproduce this crash by itself; a small test hook or a factored helper would make the zero-inode case testable.
• Keep the existing check_disk_resource_limit() / check_need_evict_cache_in_advance() coverage for high space and high inode percentages, since those paths should still trigger resource-limit or early-eviction behavior when inode statistics are available.

Missing information that would help define the affected environment scope:

• Filesystem type and mount options for the file-cache path, for example statfs.f_type, df -T, and the configured file cache directory.
• Whether the same cache path/filesystem is used by all BEs or only by the crashing BE.
• Which target branches are needed for the fix: master, branch-4.1, and/or the affected cloud build branch/tag.

[uchenily]

This bug is related to the fs type, in btrfs, inodes are always reported as 0.

https://git.openembedded.org/bitbake/diff/?h=2.0&id=a7f6c3e67bd9170e93b2b94676e84018faf0df91
https://mpdesouza.com/blog/btrfs-for-mere-mortals-inode-allocation/#btrfs-inodes

df -i /home/chen/Programming/doris-4.1/output/be/file_cache
文件系统       Inodes 已用I 可用I 已用I% 挂载点
/dev/nvme0n1p3      0     0     0      - /home