fs_mgr: overlayfs: check if scratch device is ext4 dedupe

Do not attempt to mount scratch r/w if it is ext4 dedupe, this causes
too much noise and troubling but innocuous error reports.

Assumption is we normally try f2fs first on all devices, we only try
ext4 first if we do not have f2fs tools, or if the existing
filesystem is ext4.  That said, we only have to check if it is ext4
dedupe during the first mount attempt, the fallback mount attempt for
ext4 is unlikely to need this checking.

Changes the output report for a retrofit DAP device from:

$ adb remount
Disabling verity for /system
[libfs_mgr]superblock s_max_mnt_count:65535,/dev/block/by-name/system_b
[libfs_mgr]check_fs(): mount(/dev/block/by-name/system_b,/mnt/scratch,ext4)=-1: Invalid argument
[libfs_mgr]Running /system/bin/e2fsck on /dev/block/sda6
[libfs_mgr]__mount(source=/dev/block/by-name/system_b,target=/mnt/scratch,type=ext4)=-1: Invalid argument
[libfs_mgr]Running /system/bin/fsck.f2fs -a /dev/block/sda6
[libfs_mgr]__mount(source=/dev/block/by-name/system_b,target=/mnt/scratch,type=f2fs)=0: Success
Using overlayfs for /system
. . .

To the more pleasant:

$ adb $BL1 remount
Disabling verity for /system
[libfs_mgr]superblock s_max_mnt_count:65535,/dev/block/by-name/system_b
[libfs_mgr]__mount(source=/dev/block/by-name/system_b,target=/mnt/scratch,type=ext4)=0: Success
[libfs_mgr]umount(/mnt/scratch)
[libfs_mgr]Running /system/bin/fsck.f2fs -a /dev/block/sda6
[libfs_mgr]__mount(source=/dev/block/by-name/system_b,target=/mnt/scratch,type=f2fs)=0: Success
Using overlayfs for /system
. . .

Signed-off-by: Mark Salyzyn <[email protected]>
Test: adb-remount-test.sh
Change-Id: Ic8c642912b1bafe0b4210c69c99a1d89fa20f204

Author: Mark Salyzyn

fs_mgr/fs_mgr_overlayfs.cpp

@@ -809,15 +809,26 @@ bool fs_mgr_overlayfs_mount_scratch(const std::string& device_path, const std::s
     entry.fs_type = mnt_type;
     if ((mnt_type == "f2fs") && !f2fs) entry.fs_type = "ext4";
     if ((mnt_type == "ext4") && !ext4) entry.fs_type = "f2fs";
-    entry.flags = MS_NOATIME;
-    if (readonly) {
-        entry.flags |= MS_RDONLY;
-    } else {
+    entry.flags = MS_NOATIME | MS_RDONLY;
+    auto mounted = true;
+    if (!readonly) {
+        if (entry.fs_type == "ext4") {
+            // check if ext4 de-dupe
+            entry.flags |= MS_RDONLY;
+            auto save_errno = errno;
+            mounted = fs_mgr_do_mount_one(entry) == 0;
+            if (mounted) {
+                mounted = !fs_mgr_has_shared_blocks(entry.mount_point, entry.blk_device);
+                fs_mgr_overlayfs_umount_scratch();
+            }
+            errno = save_errno;
+        }
+        entry.flags &= ~MS_RDONLY;
         fs_mgr_set_blk_ro(device_path, false);
     }
     entry.fs_mgr_flags.check = true;
     auto save_errno = errno;
-    auto mounted = fs_mgr_do_mount_one(entry) == 0;
+    if (mounted) mounted = fs_mgr_do_mount_one(entry) == 0;
     if (!mounted) {
         if ((entry.fs_type == "f2fs") && ext4) {
             entry.fs_type = "ext4";