zfs-2.2.9 patch set #17856
Open
zfs-2.2.9 patch set #17856
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FreeBSD provides CI-IMAGES since some time. These images are based on nuageinit, which does not support fqdn and sudo for example. So we need currently some workarounds to get it working. The FreeBSD images will be more compatible with cloud-init in some near future. Then we can remove the workaround things. These versions are used for testing: - freebsd13-4r (RELEASE) - freebsd14-3s (STABLE) - freebsd15-0c (CURRENT) Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Closes openzfs#17462
When running ztest under the CI a common failure mode is for the underlying filesystem to run out of available free space. Since the storage associated with a GitHub-hosted running is fixed, we instead create a pool and use a compressed ZFS dataset to store the ztest vdev files. This significantly increases the available capacity since the data written by ztest is highly compressible. A compression ratio of over 40:1 is conservatively achieved using the default lz4 compression. Autotrimming is enabled to ensure freed blocks are discarded from the backing cipool vdev file. Reviewed-by: Tino Reichardt <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17501
FreeBSD 13.4 is EOL since June 30, 2025. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Closes openzfs#17519
The package ksh93 is replaced by ksh now. This works for FreeBSD 13 and 14 also. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Closes openzfs#17523
Testing on CentOS Stream provides several months advance notice of changes coming to the RHEL kernel. This should help OpenZFS be proactive instead of reactive to new RHEL minor versions. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by: Carl George <[email protected]> ZFS-CI-Type: full Closes openzfs#16904 Closes openzfs#17526
The latest Debian 11 image includes bullseye-backports as a default repository in the /etc/apt/sources.list. However, this repository has gone end of life which effectively breaks the default install. We shouldn't need anything in backports so lets unconditionally remove backports on all Debian builders to resolve the issue. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17569
This check is currently limited to checking mismatches that occur in the same stack frame. It does not detect across stack frames. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Richard Yao <[email protected]> Closes openzfs#17352
Adjust the regexes to match the test line with timestamps, then remove them for the summary. The internal timestamp is still in the full logs. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17045
Chase URL change from the FreeBSD project. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Colin Percival <[email protected]> Closes openzfs#17617
In the past there have been times when we need to generate new RPMs for an existing ZFS release. Typically this happens when a new RHEL version comes out and the kernel symbols no longer match. To get users to auto-update we just bump the patch number. For example, we had to create zfs-2.1.13-1 for EL8.8 and zfs-2.1.13-2 for EL8.9. This commit adds an optional patch level text box to the github package builder runner. In addition, this commit also uses `hostnamectl` instead of `hostname` for F42+ compatibility, if available. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17638
This commit adds Debian 13 alias Trixie to the checked operating systems. The image needs to be run with UEFI support. Current Debian version overview: - Debian 11 (Bullseye) -> "oldoldstable" - Debian 12 (Bookworm) -> "oldstable" - Debian 13 (Trixie) -> new "stable" The CI will be run on Debian 12 and Debian 13 now. Debian 11 is kept, but won't be used automatically. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Closes openzfs#17648
We've seen Fedora 42 still setting up after 10 min. Change the timeout to 15 min. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17697
Because GitHub creates a merge commit on top of real head, so the check on HEAD will fail regardlessly. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#17695
Otherwise it might become `if [ == "" ]` which is ill-formed. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#17695
- Increase qemu-1-setup.sh timeout to 20min since it sometimes fails to complete after 15min. - Timestamp all qemu-1-setup.sh lines to look for hangs. - Add a 'watchdog' process to print out the top running process every 30sec to help with debugging. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17714
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Closes openzfs#17749
When updating a Fedora instance to an experimental kernel make sure to include the matching versioned perf and bpftool packages. This helps ensure there are no unexpected conflicts which would prevent the new packages from being installed. Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17791
The Buildbot CI infrastructure has been fully replaced by GitHub Actions. Remove any lingering references from the repository. Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17794
Signed-off-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Closes openzfs#17795
Add a -O option to zfs-test.sh to dump debug information on test timeout. The debug info includes: - 30 lines from 'top' - /proc/<PID>/stack output of process with highest CPU usage - Last lines strace-ing process with highest CPU usage - /proc/sysrq-trigger kernel stack traces All debug information gets dumped to /dev/kmsg (Linux only). In addition, print out the VM console lines from the "Setup Testing Machines" step. We have often see VMs timeout at this step and don't know why. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17753
Signed-off-by: Shreshth Srivastava <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Tony Hutter <[email protected]>
Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Closes openzfs#17815
FreeBSD 15.0-ALPHA5 image fails to boot on cloud VMs due to missing /boot/efi mount point, causing the system to drop to single user mode where SSH cannot start. Work around this by staying on ALPHA4 and setting IGNORE_OSVERSION=yes to bypass pkg's kernel version mismatch prompt during bootstrap. This allows CI to proceed with ALPHA4 until we have a stable FreeBSD 15.0 image. Signed-off-by: Ameer Hamza <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Closes openzfs#17846
Linux 5.16 by default fails the build on objtool warnings. We have known and understood objtool warnings we can't fix without involving Linux maintainers. To work around this we introduce an objtool wrapper script which removes the `--Werror` flag. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Attila Fülöp <[email protected]> Closes openzfs#17456
Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17443
Avoid calling dbuf_evict_one() from memory reclaim contexts (e.g. Linux kswapd, FreeBSD pagedaemon). This prevents deadlock caused by reclaim threads waiting for the dbuf hash lock in the call sequence: dbuf_evict_one -> dbuf_destroy -> arc_buf_destroy Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Kaitlin Hoang <[email protected]> Closes openzfs#17561
03987f7 (openzfs#16069) added a workaround to get the blk-mq hardware context for older kernels that don't cache it in the struct request. However, this workaround appears to be incomplete. In 4.19, the rq data context is optional. If its not initialised, then the cached rq->cpu will be -1, and so using it to index into mq_map causes a crash. Given that the upstream 4.19 is now in extended LTS and rarely seen, RHEL8 4.18+ has long carried "modern" blk-mq support, and the cached hardware context has been available since 5.1, I'm not going to huge lengths to get queue selection correct for the very few people that are likely to feel it. To that end, we simply call raw_smp_processor_id() to get a valid CPU id and use that instead. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Paul Dagnelie <[email protected]> Signed-off-by: Rob Norris <[email protected]> Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Closes openzfs#17597
Update the META file to reflect compatibility with the 6.16 kernel. Tested with 6.16.0-0-stable of Alpine Linux edge, see <https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/87929>. Reviewed-by: Rob Norris <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Achill Gilgenast <[email protected]> Closes openzfs#17578
We only have extremely narrow uses, so move it all into a single function that does only what we need, with and without d_set_d_op(). Sponsored-by: https://despairlabs.com/sponsor/ Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17621
Three cases were discovered where 'zpool add' would fail to warn when adding vdevs to a pool with a mismatched replication level. These are: 1. When a pool contains mixed file and disk vdevs. 2. When a pool contains an active dRAID distributed spare 3. When a pool contains an active hot spare The lack of warnings are caused by get_replication() assessing the current pool configuration an inconsistent and disabling the mismatched replication check for the new pool configuration after 'zpool add'. This change updates get_replication() to be slightly more tolerant in the non-fatal case. The zpool_add_010_pos.ksh test case was split in to separate tests: zpool_add_warn_create.ksh, pool_add_warn_degraded.ksh, and zpool_add_warn_removal. These test were extended to include coverage for dRAID pools and the three scenarios described above. Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17780
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Attila Fülöp <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#17633 Closes openzfs#17646
tonyhutter
force-pushed
the
zfs-2.2.9-hutter
branch
from
d0aee31 to
b79f96f
Compare
Member
|
Just for a note, we have this PR open for 2.2: #17583 . I wonder if it would be cleaner to include the dependencies (if possible) rather than rewrite, though I don't have significant interest in this branch. |
Contributor
Author
- When receiving memory pressure signal from OS be more strict trying to free some memory. Otherwise kernel may come again and request much more. Return as result how much arc_c was actually reduced due to this request, that may be less than requested. - On Linux when receiving direct reclaim from some file system (that may be ZFS) instead of ignoring request completely, just shrink the ARC, but do not wait for eviction. Waiting there may cause deadlock. Ignoring it as before may put extra pressure on other caches and/or swap, and cause OOM if nothing help. While not waiting may result in more ARC evicted later, and may be too late if OOM killer activate right now, but I hope it to be better than doing nothing at all. - On Linux set arc_no_grow before waiting for reclaim, not after, or it may grow back while we are waiting. - On Linux add new parameter zfs_arc_shrinker_seeks to balance ARC eviction cost, relative to page cache and other subsystems. - Slightly update Linux arc_set_sys_free() math for new kernels. Signed-off-by: Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Reviewed-by: Rob Norris <[email protected]> Reviewed-by: Tony Hutter <[email protected]>
Previous code pruned 10% of dnodes once 3/4 of metadata appeared unevictable. On workloads with many millions of dnodes and little other metadata it creates significant load spikes for many seconds straight. This change instead gradually increases pruning as unevictable metadata grow above the 3/4, which may allow it to stabilize at some level. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#16401
Linux kernel shrinker in the context of null/root memcg does not scan dentry and inode caches added by a task running in non-root memcg. For ZFS this means that dnode cache routinely overflows, evicting valuable meta/data and putting additional memory pressure on the system. This patch restores zfs_prune_aliases as fallback when the kernel shrinker does nothing, enabling zfs to actually free dnodes. Moreover, it (indirectly) calls arc_evict when dnode_size > dnode_limit. Reviewed-by: Rob Norris <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Gionatan Danti <[email protected]> Closes openzfs#17487 Closes openzfs#17542
Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: omni <[email protected]> Closes openzfs#16164
Artix Linux is systemd free distribution based on Arch Linux, with openrc dinit runit s6 as init alternatives. This patch will make init scripts installation work the way Gentoo Linux with openrc. The scripts tweaking for other init will be left to packager. Signed-off-by: Yongming Zhao <[email protected]> Reviewed-by: Rob Norris <[email protected]> Reviewed-by: Tony Hutter <[email protected]>
Reviewed-by: Tino Reichardt <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: poscat <[email protected]> Closes openzfs#16861
Looking on txg_wait_synced(, 0) I've noticed that it always syncs 5 TXGs: 3 TXG_CONCURRENT_STATES + 2 TXG_DEFER_SIZE. But in case of dmu_offset_next() we do not care about deferred frees. And even concurrent TXGs we might not need sync all 3 if the dnode was not dirtied in last few TXGs. This patch makes dmu_offset_next() to sync one TXG at a time until the dnode is clean, but no more than 3 TXG_CONCURRENT_STATES times. My tests with random simultaneous writes and seeks over many files on HDD pool show 7-14% performance increase. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Rob Norris <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#17434
Fixes an obvious typo, where a variable was missing the required leading dollar sign ($) Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Martin Rüegg <[email protected]> Closes openzfs#17480
Use statx to verify that path-based unmounts proceed only if the mountpoint reported by statx matches the MNTTAB entry reported by libzfs, aborting the operation if they differ. Align `zfs umount /path` behavior with `zfs umount dataset`. Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Ameer Hamza <[email protected]> Closes openzfs#17481
While booting, only the needed 256KiB benchmarks are done now. The delay for checking all checksums occurs when requested via: - Linux: cat /proc/spl/kstat/zfs/chksum_bench - FreeBSD: sysctl kstat.zfs.misc.chksum_bench Reported by: Lahiru Gunathilake <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Co-authored-by: Colin Percival <[email protected]> Closes openzfs#17563 Closes openzfs#17560
tonyhutter
force-pushed
the
zfs-2.2.9-hutter
branch
from
b79f96f to
6ac7160
Compare
Contributor
Author
|
@amotin I was cherry picked the commits directly and they are included in my latest push. @lzsaver I pulled all those with the exception of:
|
We need to specifically use the FX_XFLAG_* macros in zpl_ioctl_*attr() codepaths, and the FS_*_FL macros in the zpl_ioctl_*flags() codepaths. The earlier code just assumes the FS_*_FL macros for both codepaths. The 6.17 kernel add a bitmask check in copy_fsxattr_from_user() that exposed this error via failing 'projectquota' ZTS tests. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17884 Closes openzfs#17869
- don't include foreign machine assembly files - reduce diff to FreeBSD module Makefile Discovered in FreeBSD port filesystems/openzfs-kmod Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by: Martin Matuska <[email protected]> Closes openzfs#17219
Disable the aarch64 NEON SIMD intrinsics for kernel builds. Safely using them in the kernel context requires saving/restoring the FPU registers which is not currently done. Additionally, remove the aarch64 optimized PREFETCH_L1 and PREFETCH_L2 instruction. Rely on the more portable compiler built ins. This lets us remove the problematic workaround in the aarch64_compat.h header which undefines the __aarch64__ macro. Reviewed-by: Rob Norris <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17904 Closes openzfs#17852
Update FreeBSD versions: - add FreeBSD 15.0-STABLE - add FreeBSD 16.0-CURRENT So we use the latest versions of each line now: - Freebsd 14.3 (RELEASE) - FreeBSD 15.0 (STABLE) - FreeBSD 16.0 (CURRENT) In commits - you may specify which type of CI should run: - ZFS-CI-Type: quick - ZFS-CI-Type: linux - ZFS-CI-Type: freebsd - ZFS-CI-Type: full Reviewed-by: Alexx Saver <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Closes openzfs#17896
BRT_RANGESIZE_TO_NBLOCKS() takes number of ranges as its argument. To get number of blocks we should multiply it by the entry size, not divide by it, as it was due to missing parentheses. Before openzfs#17875 this could cause small memory corruptions for vdevs bigger than 64TB, but the change made the bug more noticeable. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Closes openzfs#17886 Closes openzfs#17915
META file and changelog updated. Signed-off-by: Tony Hutter <[email protected]>
tonyhutter
force-pushed
the
zfs-2.2.9-hutter
branch
from
6ac7160 to
079ba86
Compare
behlendorf
approved these changes
Nov 13, 2025
behlendorf
added
Status: Accepted
amotin
approved these changes
Nov 13, 2025
|
Yes, you are right. Also #17563 has problems at least on Gentoo. We need to figure it out. Let's skip it for now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
22 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Proposed patch set for zfs-2.2.9.
Description
Add 6.17 kernel support, and a few fixes. Also update CI code to latest version.
How Has This Been Tested?
CI will test
Types of changes
Checklist:
Signed-off-by.