Can't boot TWRP with Nougat's file-based encryption [bullhead] #854
Comments
|
I agree. The file-based encryption support needs to be extended from the Pixels to ALL 2016+ Nexus devices, including the 5X (bullhead), 6P (angler), and Pixel C. |
|
TWRP 3.1.1 boots for me, but naturally FBE isn't getting decrypted since support for that in TWRP hasn't been added to bullhead or angler yet. |
|
@osm0sis It would be great if it was added. FBE is a very useful feature. Also, it states under the changelog that there is "FBE Support" - it should be clarified to state that decryption only works on the Pixel. |
|
Good to know, I suppose I gotta wait a bit longer with the clean upgrade. I wonder if it's not being missed by some bullhead-owning developer :) |
|
Update from me contacting Dees_Troy here: https://forum.xda-developers.com/nexus-5x/orig-development/recovery-twrp-2-8-7-0-touch-recovery-t3230471/post73542155#post73542155 I wonder if @gubacsek would be able to examine the state of things like he did on Pixel. |
|
@osm0sis Nice :D, if there's anything I can do to help aswell that'll be great. Currently my Telegram is unavailable and I also don't have my phone to test on atm - but any "Deaktop PC" work I can try do. PS - I was quoted in that post and didn't notice :P |
|
@osm0sis I have a nexus 6p and would be interested in helping. I've already got a build environment set up (was trying to get fbe working myself) and I just downloaded the pixel diff, gonna try integrating some of those changes. Edit: Hey guys, I got it working! Gonna be cleaning things up a bit then uploading, same method should work for the 5x. |
|
Everything seems to be working well, uploaded the device trees on my profile but they rely on some changes in other projects atm. Here's an image for the 6p (twrp-angler-fbe.zip) along with one for the 5x if someone else would like to test it (twrp-bullhead-fbe.zip). |
|
@chetgurevitch niceee I previously had it working in an older version of Android but thx for this - only issue now is to get my ROM to support FBE again :P PS - i'll try test it out within the coming week (5x version). PPS - Would be nice if we could see every change done but its great enough that you did this. btw do u have XDA account? If yes, go upload it there too - I'm sure people would appreciate it. |
|
The 5X build is working perfectly so far! Thanks so much! Hopefully your changes get rolled into the official build soon! Have/can you posted them on the official Gerrit? Whew! Good to have a working recovery again! 😃 |
|
@osm0sis Not yet. FBE depends on some changes in other projects than the device tree that I pulled from the pixel diff @gubacsek posted on xda. I'm guessing those will land in the upstream projects as the Oreo bringup progresses but that could use some clarification. Anyways here's a zip containing a patch with all the necessary changes outside of the device tree and some scripts to apply or revert repo diffs if you wanna try building it yourself. Also @followmsi I think we can probably get fbe working on the pixel C. It might be as simple as applying the diff above, copying the pulldecryptfiles.sh and init.recovery tweaks from here and changing the block device names. Edit: Also everything seems to work fine on my 6p whether using a nuget kernel or an oreo kernel, including adb sideload. The images and device trees I uploaded use googles prebuilt kernels for android-7.1.2_r29, which is the tag omnirom nuget sources are on. |
|
Thanks for the infos .. I will have a look. Still fighting with other strange Oreo issues .. And no time to check and solve all the issues :) EDIT: .. but your qseecomd solution should make FBE working on flo/deb devices too :) |
|
@chetgurevitch it seems like @Dees-Troy's Oreo/Pixel FBE bringup is complete now, any chance you could submit your changes for bullhead/angler officially now? |
|
I'll look into this in the coming days. I was able to trim all but two of the changes in the patch set I linked and successfully decrypt but I'm currently running magisk which disabled my encryption and I'm not sure how it will behave when I attempt to re enable it so I'll have to back everything up beforehand. I'll also look into how magisk can/does handle FBE on older devices. |
|
Magisk works just fine with FBE on my Nexus 6P. It uses the cache partition, which is not encrypted. If you flash the standard zip, you do have to re-enable dm-verity in fstab or it won't boot. If you flash from the app I think it correctly passes the option to the script to enable it. |
|
@morganw3 Awhile back I somehow ended up with a data partition configured for fbe which was then disabled by magisk and a functinal device. Anyways backed up and reinstalled everything, set the KEEPFORCEENCRYPT flag in /data/.magisk and everthing worked fine. I've also been updating my angler device tree. Dropped pulldecryptfiles.sh in favor of bundling the necessary binaries again and only the install_keyring patch is necessary. Had everything ready to go yesterday but I just updated to 8.1 and it doesn't work ¯\_(ツ)_/¯ TWRP built with the kernel and binaries from 8.1.0_r1 was able to decrypt successfully when I was still on 8.0 and before I booted 8.1 when I was flashing magisk. Currently investigating. |
|
Just updated to 8.1 here myself to find the 3.1.1-FBE @chetgurevitch built doesn't work anymore (shows the pattern screen to unlock but then no pattern works). Hopefully something can be figured out! 😃 |
|
Oh wait, interesting! Official TWRP 3.2.1 doesn't decrypt anything (/data and /sdcard are both decrypted), but I found the 3.1.1-FBE actually decrypts most things in /data (but not /sdcard) despite the failed pattern. Better than nothing so I guess I'll stick with that until a full decrypt can be worked out. 👍 |
|
@chetgurevitch, @gubacsek, check out codeworkx's 'universal' FBE OP5T build changes, might have what we need here: https://forum.xda-developers.com/oneplus-5t/development/recovery-twrp-3-2-1-0-oreo-8-0-8-1-t3729673 |
|
@osm0sis Thanks, the holdup was actually gatekeeper based synthetic password support in twrp. I'd made little progress on that locally but turns out @Dees-Troy pushed the necessary changes to git a couple days ago and I've got a working build now. Just gotta cleanup the device trees then I'll try and get this upstreamed. |
|
@chetgurevitch I hate to ask things like this; I know it gets annoying. Could you possibly upload the img you built to a hosting site or the device tree to GitHub? I tried working with the repo you have on your profile but didn't get anywhere :) |
|
@harryyoud Updated the angler device tree on github. Requires fbe-patch.zip twrp-3.2.1-0-angler-fbe.zip built using the twrp-8.1 manifest from here. USB-OTG seems to be broken. |
|
@osm0sis got a bullhead test image if you want to try it out twrp-3.2.1-0-bullhead-fbe.zip. Also updated my bullhead device tree. |
|
Thanks! FBE decryption working great, but a slight regression with "Updating partition details..." taking a long time each time it happens. |
|
@chetgurevitch Thanks so much! That works great here on LineageOS 15.1 for angler (based on AOSP 8.1r4). Gonna see if I can get it self built now |
|
@chetgurevitch I'm trying to convert to FBE but whenever I do that, it boots into TWRP and asks me for a password to decrypt the data. Only problem is that I haven't entered any password at any point. Using the bullhead 3.2.1-0 test image. EDIT: Managed to get encrypted with FBE and have TWRP (I'm lacking the RPMB partition so it was a bit tricky) but when I go to TWRP it doesn't ask to decrypt and only shows scrambled folder names. |
|
Here's a snip from the recovery.log. |
|
@CazeW I'm not super familliar with the RPMB issue and I don't own a nexus 5x to test stuff on. The initial conversion should be done with a full stock stack though, including recovery. |
|
@chetgurevitch Yeah, I figured that out after trial and error. The RPMB issue is basicly that the bootloader stays unlocked only until reboot. Reboot to bootloader or start normally and it's immediately locked again, which obviouly causes /data to be wiped. I managed to get everything else working but for some reason your TWRP build isn't decrypting /data for me like it should (like it shows in that log). |
|
Because of a stupid mistake I did when updating Magisk, I had to rewipe everything and start from scratch. To my surprise though, TWRP is now working and decrypting my /data correctly. The only difference I could think of was when I first booted the phone, I set it to ask for the password at boot before converting to FBE. |
|
@Lothsahn @DavidBerdik @pongo1231 I've made a test image for shamu but it ends up larger than the devices recovery partition. I'm uploading an uncompressed (actually gzipped but eh) and xz compressed image, the compressed one may fail to boot since I'm not sure the lineageos kernel includes xz support. If this is the case it may be possible to boot the uncompressed image using fastboot boot rather than writing it to the device, though I'm not sure on that either. compressed-twrp-shamu.img.zip |
|
@chetgurevitch Thank you for your effort, although I'm really bummed about it being too big. By how much does your build exceed the partition limitations? I am not sure what the limit is on the Nexus 6, and a quick Google search did not reveal that information to me. Using fastboot boot may be a suitable temporary workaround, but in the long run, it would be nice to either cut the image down to size or expand the recovery partition so that it can fit. I am currently in the middle of finals so I don't have time to play with this, but once they're over, perhaps I can dedicate a little time to looking into doing one of these two things. I am not very knowledgeable about how Android partitions are handled though, so I may not get very far. |
|
@chetgurevitch I finally tried using your two images today, and unfortunately, neither one worked. The compressed one does not work, and the uncompressed one obviously does not flash since it's too big. I tried using "fastboot boot", but apparently it's a feature that is no longer present in adb. |
|
@DavidBerdik |
|
@tlwhitec If it is still present, then I most definitely did not run it properly. I was booted in to the bootloader though. How is it supposed to be used? I used |
|
@DavidBerdik yes, but YMMV, apparently. Try that and you'll see ;) |
|
@chetgurevitch The compressed flashed and then entered a constant reboot loop. It never got past the bootloaders The uncompressed could not flash: It's actually not FBE in LOS 15.1. There's some change in the full disk encryption in recent builds that cause failures. I'm working to try to port the Android FDE code from LOS 15.1 into TWRP, but it's obviously not an easy process. |
|
@tlwhitec Okay, I've tried that and unfortunately I got the same result as @Lothsahn. To verify if fastboot boot is working properly for me, I tried doing it with an older version of TWRP, and it worked, so it's an issue with the images. @Lothsahn - Good luck to you. I am trying to put some effort into it, but I haven't been able to turn up anything useful so far. |
|
Today I tried the new TWRP release 3.2.2-0 and I'm back again to square one. The decryption fails. @chetgurevitch could it happen you didn't make a pull-request with whatever you made for the last working builds of yours? |
|
@tlwhitec I'm seeing the same thing on shamu. :( |
|
Also having the decryption problem with 3.2.2-0 on vince (xiaomi redmi 5 plus) running a pixel rom. |
|
I am experiencing the same or a similar problem on my hammerhead device running Resurrection Remix. My RR version is a recent weekly build, 6.1.0-20180701, which is based on Android 8.1. Since I encrypted the storage, I too have been unable utilize the full feature set of TWRP. I can navigate through the menus if I cancel the password prompt, but I cannot use any features that depend on decrypting the data. Upon booting TWRP, I am prompted for a password (i.e. character sequence, not pin or pattern), even though my chosen unlocking method is pattern. The situation improves slightly if I enable, through the Android settings, the option to require unlocking at the time of boot (for behavior similar to the pre-nougat FDE security). In this case, TWRP correctly identifies pattern unlocking as the chosen scheme, but does not accept the pattern I have chosen through the Android settings. Such behaviors are the same using the official builds for both TWRP 3.2.1-2 and 3.2.2-0 on hammerhead. I am aware that some unofficial builds are available for Pixel models that work better with newer encryption schemes, but as my device is not a Pixel, I am not able to try to these builds on my device. |
|
@brainchild0 There are unofficial builds for nexus 5x in this thread, scroll up to march 12th. I haven't made a pull request for a few reasons although the only blocking one would be that I haven't confirmed backup and restore work properly on both fbe and fde systems. The other 2 are updating partition details and mtp storage though like I said I don't consider either of them blocking. I'm not really in a hurry to test the backup and restore on my own device though, especially considering this is an officially unsupported feature for devices that have less than 5 months of support left on them. If someone does the testing I'll try to move forward with upstreaming. |
|
@chetgurevitch Have you put any additional effort into trying to get the Nexus 6 build down to size? |
|
@chetgurevitch I am using Nexus 5, not Nexus 5x. |
|
@brainchild0 hammerhead (Nexus 5) does not support File-Based Encryption (FBE). bullhead (Nexus 5X) and angler (Nexus 6P) do, and they are what this thread is about. You should open another issue or seek help in your ROM thread on XDA. |
|
@osm0sis @chetgurevitch Please correct me if I am in error, but I understand that support for FBE is a software feature of the OS. I further understand that while some devices have hardware features to facilitate security and accelerate I/O while encryption is use, any device that is able to run a version and build of Android that supports FBE is itself able to utilize FBE. On my device currently, the system boots to the lock screen and displays notifications from my installed applications before any unlocking code is applied. This behavior is is stark contrast to the earlier FDE mode, in which loading any user-installed applications was delayed until after the user provided the unlock code, through an interface slightly different and less elaborate than the standard lock screen. I understand that this difference was one of the central objectives in the development of the FBE feature. It may be so that official stock builds for certain devices do not include support for FBE, but because I am using Resurrection Remix, this disability would not apply to me unless the feature was also omitted from this build. As such, according to my best understanding and current observations, I am in fact using FBE on my Nexus 5. Even if not, the issue still remains that TWRP cannot decrypt the data on my device, for whatever reason not yet identified. Again, please correct any error in my understanding. |
|
I am sorry if it would appear that I am attempting to hijack the thread, but I feel I should report that I have investigated further, and become quite confused. The below article, from the developer, describes the Direct Boot feature of FBE, which I described above and observe on my device. However, when I follow the instructions to verify encryption type, the results indicate FDE, as @osm0sis suggests I should expect. Perhaps this discrepancy points to some inconsistent state, preventing TWRP from decrypting the data? Article: |
|
Hi @chetgurevitch, thanks for your continuous support :) In fact I don't use FBE yet, but apparently some kind of FDE where it doesn't require me to enter anything during bootup. Even the TWRP of yours doesn't require me to enter any pattern or code, while the The latest official TWRP 3.2.2-0 asks me for password (where it accepts nothing I enter), so the device is indeed somehow encrypted, but I apparently lost track how it works these days (on Android 8.1 based LineageOS 15.1). Seems like @brainchild0 has the same observation for hammerhead on an 8.1 based OS, so that could be actually common among the devices. Regarding your offer, I'd love to contribute helping moving this forward by testing my FDE part, but the last TWRP release has a note about The only backup-related change I could find was |
|
@tlwhitec The main reason for "no passcode/pin/pattern on boot" is due to an active (turned on) Accessibility service. This is to make sure the service loads for the user who needs ease-of-access of the device from the moment it has switched on. Another reason may be it's using internal/device keys to be encrypted rather than your passcode/pin/pattern I'm not sure how to correct this one but custom ROMs and a data wipe while setting up with a pin/passcode/pattern have always avoided it for me. |
|
For a Nexus 5X bullhead with FBE and latest Android 8.1.0 patches, which unofficial TWRP build should be used? Not sure whether there is a build based on twrp-3.2.2-0 |
|
Still this one: #854 (comment) No 3.2.2 one has been done yet. |
|
I am offering an update, being, unfortunately, unable to provide good news, and wanting to provide clarification following my previous, confused comments. I confirmed through ADB that the encryption scheme, as suggested, is FDE (confusing because I see no reason why encryption method depends on hardware device). However, I remain unable to use most features of TWRP, because of inability to mount the data partitions. On start, TWRP prompts for a password, regardless of whether a password is set, but never accepts any given password. I have tried version 3.2.3, seeing no change on this problem from 3.2.2. Currently, I cannot upgrade, maintain, or even backup the OS. The only options are to keep the current installation or to reset the device by clearing all data. If any partial solution or workaround is currently available, it would help me to move forward in advance of full resolution. Has a patched version been built for hammerhead, and if so, is running it like to resolve the issue I am describing? To summarize: |
I am having the same issue with the same device. Device: LG Nexus 5 (Hammerhead) D820 |
|
Guys this is the FIle-Based Encryption (FBE) issue thread, for bullhead (Nexus 5X), and also by extension angler (Nexus 6P). If you're having problems with Full Disk Encryption (FDE) on some other device this is not the place for you. |
|
Based of this comment. by @laijirong: TeamWin/android_device_huawei_angler#6 (comment) For the nexus 6P (angler), we might wish to compare this fstab: To this fstab, by @Jertlok: Which is apparently created for this custom rom: https://forum.xda-developers.com/nexus-6p/development/rom-statixos-t3832438 However, that custom rom & the recovery they make also seems to switch SELinux to permissive, which I really don't want. |
|
Do we know whether the official 3.3.0 supports FBE? |
|
I haven't tested yet, but I don't believe it does since I saw someone mention wanting support added still. |
|
Just tried it out on my bullhead. No idea about FBE because 3.3.0 won't even boot. |
Dear TWRP gurus,
The thing is quite simple, after installing the stock 7.1 image [n4f26o Feb 2017] on my "new" bullhead and converting it to file-based encryption, TWRP refuses to finish booting after flashing. For a moment it displays some errors about failing to decrypt /data and then reboots, then (the bootloader gets magically locked and) TWRP boots again, with the same problem. This all over again.
The text was updated successfully, but these errors were encountered: