Integrate Termux Shell & Terminal Activity.

[cryptrr]

This will change the default shell from sh to bash

Will install termux-bootstrap packages by default.

Better package management (potentially with some hacks)

AutoPie gets a full fledged terminal and the possibility to build packages.

[robertkirkman]

I have good news, my most recent CI workflow run for the prototype bootstrap you wanted to test completed, so termux-generator now works fully in CI,

https://github.com/robertkirkman/termux-generator/actions/runs/16575505695

It'll be a fully working APK, but you can just unzip the APK and open the assets folder to get the bootstrap if that is the only part you wanted to start with.

I'm going to be generating some more bootstraps and making other changes to termux-generator while I start to try to edit AutoPie's code. If you want to try generating more bootstraps/APKs with termux-generator yourself, you can fork the repository and enable GitHub Actions on your fork, then go to the GitHub Actions tab and find the GUI for termux-generator.

The way to try installing clang to the bootstrap is basically that it's necessary to write the name "libllvm" into the package add list instead of "clang" or "build-essential". using the most top-level parent package name works around bugs in F-Droid Termux's bootstrap generator code. That won't actually work at the moment since I am changing some things in it, but it will after a while.

[cryptrr]

Awesome! Now I can start integrating it.

I compressed the bootstrap.zip into tar.xz and it reduced the file size to 139 MB.

it's necessary to write the name "libllvm"

Thanks for the heads up. I would have used build-essential if not for that and would have wasted hours.

[cryptrr]

Extremely weird error here

When creating the bash shell

CANNOT LINK EXECUTABLE "/data/user/0/com.autosec.pie/files/usr/bin/bash": library "libreadline.so.8" not found: needed by main executable

I checked the libs and there is no libreadline.so.8 but only libreadline.so.8.3

I'm now trying setting the symlinks from "SYMLINKS.txt"

[robertkirkman]

Extremely weird error here

When creating the bash shell

CANNOT LINK EXECUTABLE "/data/user/0/com.autosec.pie/files/usr/bin/bash": library "libreadline.so.8" not found: needed by main executable

I checked the libs and there is no libreadline.so.8 but only libreadline.so.8.3

I'm now trying setting the symlinks from "SYMLINKS.txt"

Correct, that's right - when using the default Termux APK, the Termux's APK's Java code extracts the bootstrap in a kind of special way - it reads the SYMLINKS.txt and creates files like libreadline.so.8 that are symbolic links to other libraries.

Here's the part of Termux APK's code that does that:

https://github.com/termux/termux-app/blob/7bceab88e2272f961d1b94ef736f1a9e20173247/app/src/main/java/com/termux/app/TermuxInstaller.java#L163

The basic reason that system is set up this way is because .zip files don't natively support symbolic links, so they're stored in this alternative format while inside the bootstrap.

Since you said you're able to use a .tar.xz bootstrap instead of a .zip, this means that if we refactor the way the bootstrap is generated to support more kinds of bootstrap better, we could hypothetically get rid of SYMLINKS.txt for .tar.xz bootstraps and store the symbolic links natively as .tar.xz, which supports them.

My plan has also been to eventually support .7z bootstraps with termux-generator too, since .7z (LZMA) is technically the # 1 space-optimal compression format, and that would fit even more bootstrap packages in the same space as .tar.xz, but I just haven't gotten around to that yet.

[robertkirkman]

I realized that clang (also known as libllvm) takes 5 hours and 30 minutes to compile inside of GitHub Actions, so it's unfortunately not really possible to build an entire custom bootstrap with it and everything else inside just in GitHub Actions without splitting it up into lots of smaller workflow runs (like official Termux does),

but if you want to get the bootstrap variant with clang + everything else all preinstalled, which is about 1 GB as a.zip, you should currently be able to get that locally from this command, since I've now put fixes in termux-generator to make it up-to-date for this command for the time being:

./build-termux.sh --name com.autosec.pie \
                  --add make,pkg-config,autoconf,automake,bc,bison,cmake,flex,gperf,libtool,libllvm,m4,git,golang,nodejs,patchelf,python,ruby,rust,subversion,python,proot-distro,ffmpeg \
                  --architectures aarch64

[robertkirkman]

Keep in mind that it's 1 GB because there is actually a ton of extra stuff in there that isn't necessarily that important, and also, in that example I did add some additional packages for cutting-edge features that are great, but do take up a lot of space, like the rust and nodejs and golang packages. Eventually I will try implementing more bootstrap size optimizations in termux-generator, which will bring the overall size of the bootstrap down, even when there are a lot of packages in it.

[cryptrr]

https://github.com/cryptrr/AutoPie/blob/main/app/src/main/assets/env.sh

I changed the paths to reflect the new folder structure but some things are elusive.

Making this work is mostly depended upon setting the ENV variables just right.
The above are the env variables I am using currently. Is there termux specific env varibles I need to set for the termux's bash to correctly init? Critical ones?

like PREFIX or TERMUX_PREFIX etc?

Or some other peculiarity while termux sets up its shell? I don't want to miss anything.

Also, I noticed this TERMUX_STAGING_PREFIX_DIR_PATH

Is the installation env actually staged somewhere before moved to the real location?

Right now, I'm trying to port all the logic from the TermuxInstaller class.

[cryptrr]

Also, If you have already made a very minimal bootstrap package. WIthout python clang, ffmpeg or anything.
Do send me one because extracting itself is taking a fair bit long while installing the app.

[robertkirkman]

https://github.com/cryptrr/AutoPie/blob/main/app/src/main/assets/env.sh

Making this work is mostly depended upon setting the ENV variables just right. The above are the env variables I am using currently. Is there termux specific env varibles I need to set for the termux's bash to correctly init? Critical ones?

like PREFIX or TERMUX_PREFIX etc?

Yeah, this part is a bit fiddly. This is the part that I was preparing to work on when I get to the part of editing AutoPie's code. There was a specific way I was planning to try to set this part up, but you can definitely go ahead if you have an idea for how you want to try to do it! I can also make my own variants gradually, to test what's possible.

I would recommend just replacing all of AutoPie's variables with the variables that Termux App uses to launch its bootstrap, except for any variables that you know are AutoPie-specific that need to be set in the shell that AutoPie runs.

You should be able to check what other variables Termux App normally sets, in these areas of Termux's code:

(a noninteractive 'single-command-runner' part of Termux's code)

https://github.com/termux/termux-app/blob/7bceab88e2272f961d1b94ef736f1a9e20173247/termux-shared/src/main/java/com/termux/shared/shell/command/runner/app/AppShell.java#L108

(the actual, primary interactive terminal part of Termux's code)

https://github.com/termux/termux-app/blob/7bceab88e2272f961d1b94ef736f1a9e20173247/termux-shared/src/main/java/com/termux/shared/termux/shell/command/runner/terminal/TermuxSession.java#L137

[robertkirkman]

Also, you can see that at this nearby section, Termux App sets the default shell to start in an order of precedence like this, and launches the first one that is available:

1. login
2. bash
3. zsh
4. fish
5. sh
6. /system/bin/sh

https://github.com/termux/termux-app/blob/7bceab88e2272f961d1b94ef736f1a9e20173247/termux-shared/src/main/java/com/termux/shared/termux/shell/command/runner/terminal/TermuxSession.java#L96

https://github.com/termux/termux-app/blob/master/termux-shared/src/main/java/com/termux/shared/shell/command/environment/UnixShellEnvironment.java#L56

My plan was to do something involving copying large chunks of this part of Termux into AutoPie/AutoPie into Termux to merge them together seamlessly, but you might have a better idea already.

[cryptrr]

Wonderful. I'll try to port the logic from these files into the already existing code and if it becomes untenable I'll need to start with creating entirely new classes from these.

[robertkirkman]

Out of what I mentioned in the comments above, I think one of the most important details to notice is this:

while the issue description here suggests the default shell will be switched from sh to bash, and that is basically true,

The Java code of Termux doesn't launch bash by default as its first choice, the first choice is actually "login",

this is important because login also just defaults to launching bash, but it also sets up some important Termux environment variables, and if the user runs the Termux command chsh -s zsh or a similar command, they can change the shell that the login command launches from the default bash to a different shell, which remains set even after restarting the app or the device.

There might also be other things that Termux App's Java code does that would be important for the stability of using a Termux bootstrap in AutoPie, but that is just one of the ones I currently know about that came to mind.

[robertkirkman]

Also, I noticed this TERMUX_STAGING_PREFIX_DIR_PATH

Is the installation env actually staged somewhere before moved to the real location?

Sorry, I didn't see these messages earlier while reading your other messages,

Yes, I believe it is, and there are some other small quirks about how Termux installs itself that might be a little bit tedious to reimplement if rewriting everything there.

I was thinking of trying to just copy and paste as much of that part as possible directly from Termux (and when I say copy and paste, I mean automated copy-and-paste using termux-generator so that future improvements or changes from upstream Termux are easy to apply),

but if you were thinking of taking a more rewrite-oriented approach, it might also be pretty easy to just reimplement the bare minimum of the TermuxInstaller.java to get it working, and add anything additional whenever it ends up being necessary.

Also, If you have already made a very minimal bootstrap package. WIthout python clang, ffmpeg or anything.
Do send me one because extracting itself is taking a fair bit long while installing the app.

I'll perform a CI build using termux-generator that has just a completely default bootstrap, which will actually still be bigger than the official F-Droid Termux bootstrap for now, because of a bootstrap size bug in the source code of F-Droid Termux, but it'll be a lot smaller than the 200 MB one.

This should allow you to easily test basic functionality using a smaller bootstrap.

[robertkirkman]

I've started a GitHub Actions workflow run in the termux-generator repository with just this option set and nothing else, which should apply only the patches necessary to change the entire app name to the desired name:

When finished this should give you an APK that contains four bootstraps, one for each architecture, and each one should be relatively small.

https://github.com/robertkirkman/termux-generator/actions/runs/16595844278/job/46942144507

I'm pretty sure the default package set of Termux can build 4 times in under 6 hours (once for each architecture), so this should be convenient to let you test 32-bit or Android-x86 support if desired.