Skip to content

Commit 80fdbda

Browse files
committed
Initial commit.
0 parents  commit 80fdbda

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

76 files changed

+21804
-0
lines changed

.gitignore

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
build/
2+
polyfill-glibc
3+
.ninja_deps
4+
.ninja_log
5+

README.md

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
## polyfill-glibc
2+
3+
How often have you compiled a C/C++ program on a recent Linux system, tried to run that compiled program on an older Linux system, and then hit a GLIBC version error?
4+
Concretely, perhaps you're seeing something like this:
5+
6+
```
7+
new-system$ gcc my-program.c -o my-program
8+
old-system$ scp new-system:my-program .
9+
old-system$ ./my-program
10+
./my-program: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by ./my-program)
11+
```
12+
13+
The motivating idea behind polyfill-glibc is that an extra post-compilation step can prevent these errors from happening:
14+
15+
```
16+
new-system$ gcc my-program.c -o my-program
17+
new-system$ polyfill-glibc --target-glibc=2.17 my-program
18+
old-system$ scp new-system:my-program .
19+
old-system$ ./my-program
20+
It works!
21+
```
22+
23+
## Build and run instructions
24+
25+
To build polyfill-glibc, you'll need a git client, a C11 compiler (such as `gcc`), and [`ninja`](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages). With these tools present, the build process is:
26+
27+
```
28+
$ git clone https://github.com/corsix/polyfill-glibc.git
29+
$ cd polyfill-glibc
30+
$ ninja polyfill-glibc
31+
```
32+
33+
Once built, running it is intended to be as simple as passing the oldest version of glibc you want to support, along with the path to the program (or shared library) to modify, as in:
34+
```
35+
$ ./polyfill-glibc --target-glibc=2.17 /path/to/my-program
36+
```
37+
38+
If running it _isn't_ this simple, then open a GitHub issue describing why not, and we'll try to improve things.
39+
40+
Note that at present, the `--target-glibc` operation of polyfill-glibc is only implemented for x86_64. If other architectures are of interest to you, open a GitHub issue so that we can gauge demand.
41+
42+
If distributing shared libraries, polyfill-glibc can also be used to inspect dependencies (with the `--print-imports` and `--print-exports` operations), and to modify how shared libraries are loaded (with the `--set-rpath`, `--set-runpath`, and `--set-soname` operations). Consult [the documentation](docs/Command_line_options.md) for a full list of command line options.
43+
44+
## License
45+
46+
polyfill-glibc itself is made available under the [MIT license](https://opensource.org/license/mit). The polyfilling procedure can sometimes involve linking small pieces of polyfill-glibc into the file being modified; the "the above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software" clause of the MIT license is explicitly waived for any such pieces.

build.ninja

+95
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
cc_opt = -O2 -s
2+
# cc_opt = -g -O0 -fsanitize=address
3+
link_opt =
4+
5+
rule cc
6+
depfile = $out.d
7+
command = gcc -MD -MF $out.d -Wall -Wextra -Wshadow -fvisibility=hidden -fno-math-errno -ffp-contract=off -fno-trapping-math -fno-semantic-interposition -fmax-errors=5 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -std=c11 $cc_opt -c -o $out $in
8+
rule link
9+
command = gcc $cc_opt -o $out $in $link_opt
10+
rule run
11+
command = $in $out
12+
rule create_polyfill_so
13+
command = ./$in empty:$arch --add-hash --add-gnu-hash --create-polyfill-so --target-glibc=$target_glibc --output $out
14+
rule run_polyfill_glibc_txt
15+
command = ./polyfill-glibc >$out $in $link_opt
16+
rule run_polyfill_glibc
17+
command = ./polyfill-glibc --output $out $in $link_opt
18+
19+
build polyfill-glibc: link build/main.o build/renamer.o build/polyfiller.o build/x86_64/polyfiller.o build/erw.o build/v2f.o build/tokenise.o build/sht.o build/uuht.o build/arena.o build/reloc_kind.o build/common.o
20+
default polyfill-glibc
21+
build build/x86_64/polyfills.so: create_polyfill_so polyfill-glibc
22+
arch = x86_64
23+
target_glibc = 2.3.2
24+
build build/x86_64/polyfiller.o: cc src/x86_64/polyfiller.c | build/x86_64/assembled_gen.h
25+
build build/x86_64/assembled_gen.h: run build/x86_64/assembler src/x86_64/_init.s src/x86_64/c11_thread.s src/x86_64/fatal.s src/x86_64/glibc_2_6.s src/x86_64/glibc_2_7.s src/x86_64/glibc_2_15.s src/x86_64/glibc_2_16.s src/x86_64/glibc_2_18.s src/x86_64/glibc_2_25.s src/x86_64/glibc_2_26.s src/x86_64/glibc_2_30.s src/x86_64/glibc_2_31.s src/x86_64/glibc_2_34.s src/x86_64/glibc_2_36.s src/x86_64/glibc_2_38.s src/x86_64/posix_spawn.s src/x86_64/qsort.s src/x86_64/stdbit.s src/x86_64/syscalls.s src/x86_64/syscalls_ac.s
26+
build build/x86_64/assembler: link build/x86_64/assembler.o build/tokenise.o build/sht.o build/uuht.o build/common.o
27+
build build/x86_64/assembler.o: cc src/x86_64/assembler.c | build/x86_64/assembler_gen.h
28+
build build/x86_64/assembler_gen.h: run build/x86_64/build_assembler src/x86_64/insn_encoding.txt
29+
build build/x86_64/build_assembler: link build/x86_64/build_assembler.o build/tokenise.o build/sht.o
30+
build build/x86_64/build_assembler.o: cc src/x86_64/build_assembler.c
31+
build build/x86_64/renames.h: run build/parse_renames src/x86_64/renames.txt
32+
build build/polyfiller.o: cc src/polyfiller.c
33+
build build/main.o: cc src/main.c
34+
build build/renamer.o: cc src/renamer.c | build/x86_64/renames.h
35+
build build/parse_renames: link build/parse_renames.o build/tokenise.o build/sht.o build/common.o
36+
build build/parse_renames.o: cc src/parse_renames.c
37+
build build/erw.o: cc src/erw.c
38+
build build/v2f.o: cc src/v2f.c
39+
build build/tokenise.o: cc src/tokenise.c
40+
build build/sht.o: cc src/sht.c
41+
build build/uuht.o: cc src/uuht.c
42+
build build/arena.o: cc src/arena.c
43+
build build/reloc_kind.o: cc src/reloc_kind.c
44+
build build/common.o: cc src/common.c
45+
46+
build build/x86_64/test/arc4random.txt: run build/x86_64/test/arc4random build/x86_64/polyfills.so
47+
build build/x86_64/test/arc4random: link build/x86_64/test/arc4random.o build/test/syscall_filter.o
48+
link_opt = -ldl
49+
build build/x86_64/test/arc4random.o: cc test/x86_64/arc4random.c
50+
51+
build build/x86_64/test/glibc_2_38.txt: run build/x86_64/test/glibc_2_38 build/x86_64/polyfills.so
52+
build build/x86_64/test/glibc_2_38: link build/x86_64/test/glibc_2_38.o
53+
link_opt = -ldl
54+
build build/x86_64/test/glibc_2_38.o: cc test/x86_64/glibc_2_38.c
55+
56+
build build/x86_64/test/nan.txt: run build/x86_64/test/nan build/x86_64/polyfills.so
57+
build build/x86_64/test/nan: link build/x86_64/test/nan.o
58+
link_opt = -ldl
59+
build build/x86_64/test/nan.o: cc test/x86_64/nan.c
60+
61+
build build/x86_64/test/qsort_r.txt: run build/x86_64/test/qsort_r build/x86_64/polyfills.so
62+
build build/x86_64/test/qsort_r: link build/x86_64/test/qsort_r.o
63+
link_opt = -ldl
64+
build build/x86_64/test/qsort_r.o: cc test/x86_64/qsort_r.c
65+
66+
build build/x86_64/test/stdbit.txt: run build/x86_64/test/stdbit build/x86_64/polyfills.so
67+
build build/x86_64/test/stdbit: link build/x86_64/test/stdbit.o
68+
link_opt = -ldl
69+
build build/x86_64/test/stdbit.o: cc test/x86_64/stdbit.c
70+
71+
build build/x86_64/test/totalorder.txt: run build/x86_64/test/totalorder build/x86_64/polyfills.so
72+
build build/x86_64/test/totalorder: link build/x86_64/test/totalorder.o
73+
link_opt = -ldl
74+
build build/x86_64/test/totalorder.o: cc test/x86_64/totalorder.c
75+
76+
build build/x86_64/test/twalk_r.txt: run build/x86_64/test/twalk_r build/x86_64/polyfills.so
77+
build build/x86_64/test/twalk_r: link build/x86_64/test/twalk_r.o
78+
link_opt = -ldl
79+
build build/x86_64/test/twalk_r.o: cc test/x86_64/twalk_r.c
80+
81+
build test/x86_64: phony build/x86_64/test/arc4random.txt build/x86_64/test/glibc_2_38.txt build/x86_64/test/qsort_r.txt build/x86_64/test/stdbit.txt build/x86_64/test/totalorder.txt build/x86_64/test/twalk_r.txt
82+
83+
build build/test/getauxval.txt: run build/test/getauxval_pf build/test/getauxval_imp.txt build/test/getauxval_imp_pf.txt
84+
build build/test/getauxval_imp_pf.txt: run_polyfill_glibc_txt build/test/getauxval_pf | polyfill-glibc
85+
link_opt = --print-imports
86+
build build/test/getauxval_pf: run_polyfill_glibc build/test/getauxval | polyfill-glibc
87+
link_opt = --target_glibc=2.15
88+
build build/test/getauxval_imp.txt: run_polyfill_glibc_txt build/test/getauxval | polyfill-glibc
89+
link_opt = --print-imports
90+
build build/test/getauxval: link build/test/getauxval.o build/common.o
91+
build build/test/getauxval.o: cc test/getauxval.c
92+
93+
build build/test/syscall_filter.o: cc test/syscall_filter.c
94+
95+
build test: phony test/x86_64 build/test/getauxval.txt

docs/Asynchronous_cancellation.md

+79
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
## Asynchronous cancellation
2+
3+
The [`pthread_cancel`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_cancel.html) function is defective by design and should not be used. See [How to stop Linux threads cleanly](https://mazzo.li/posts/stopping-linux-threads.html) for a tour of the problem and various better solutions. That said, some applications still (misguidedly) make use of `pthread_cancel`, and some special caveats apply when polyfilling said applications.
4+
5+
## What does `pthread_cancel` do?
6+
7+
Every thread has three pieces of state:
8+
9+
|Variable name|Possible values|Initial value|Changed by|
10+
|-------------|---------------|-------------|----------|
11+
|Cancel state|ENABLE, DISABLE|ENABLE|[`pthread_setcancelstate`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_setcancelstate.html)|
12+
|Cancel type|DEFERRED, ASYNCHRONOUS|DEFERRED|[`pthread_setcanceltype`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_setcancelstate.html)|
13+
|Cancel requested|NO, YES|NO|[`pthread_cancel`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_cancel.html) (always changes to YES)|
14+
15+
A thread will be terminated (possibly with cleanup functions / destructors being called), if either of the following state combinations occur:
16+
1. _Cancel requested_ is YES, _Cancel state_ is ENABLE, _Cancel type_ is ASYNCHRONOUS.
17+
2. _Cancel requested_ is YES, _Cancel state_ is ENABLE, _Cancel type_ is DEFERRED, and the thread calls a libc function defined as a cancellation point.
18+
19+
Roughly speaking, every libc function that _could_ block execution for a non-trivial amount of time is defined as a cancellation point. This includes, but is not limited to, the following functions:
20+
* `accept`
21+
* `close`
22+
* `connect`
23+
* `copy_file_range`
24+
* `epoll_wait` / `epoll_pwait` / `epoll_pwait2`
25+
* `fcntl` (when called with `F_SETLKW` or `F_OFD_SETLKW`)
26+
* `fsync` / `fdatasync` / `sync_file_range` / `msync`
27+
* `getrandom`
28+
* `open` / `openat` / `open_by_handle_at`
29+
* `pause` / `sigpause` / `sigsuspend`
30+
* `pthread_testcancel`
31+
* `read` / `readv` / `pread`
32+
* `recv` / `recvfrom` / `recvmsg` / `recvmmsg` / `msgrcv` / `mq_receive` / `mq_timedreceive`
33+
* `send` / `sendto` / `sendmsg` / `sendmmsg` / `msgsnd` / `mq_send` / `mq_timedsend`
34+
* `sigwait` / `sigwaitinfo` / `sigtimedwait`
35+
* `sleep` / `usleep` / `nanosleep` / `clock_nanosleep`
36+
* `write` / `writev` / `pwrite`
37+
38+
## glibc semantics of cancellation points
39+
40+
Taking `epoll_pwait2` as an example, at the time of writing, the glibc implementation of `epoll_pwait2` is roughly:
41+
42+
```c
43+
old_type = pthread_setcanceltype(ASYNCHRONOUS);
44+
result = syscall(epoll_pwait2, ...);
45+
pthread_setcanceltype(old_type);
46+
check_for_pthread_cancel_race();
47+
return result;
48+
```
49+
50+
When `pthread_cancel` is called, if the target thread has _Cancel state_ of ENABLE and _Cancel type_ of ASYNCHRONOUS, then a signal is sent to the target thread. Shortly thereafter, the signal will be received by the target thread, wherein the signal handler confirms that _Cancel state_ is still ENABLE and _Cancel type_ is still ASYNCHRONOUS. If so, the thread will be terminated. If not, the signal handler will do nothing (though delivery of the signal might cause an `EINTR` result from an unrelated syscall that was being made at the time of delivery).
51+
52+
There are (at least) two issues with this implementation:
53+
1. `pthread_cancel` could send the signal _before_ `pthread_setcanceltype(old_type)`, but the signal might arrive _after_ `pthread_setcanceltype(old_type)`.
54+
2. A signal (unrelated to cancellation) could be delivered during the `syscall`, and the handler for that signal could perform a `longjmp`, thereby causing `pthread_setcanceltype(old_type)` to not be called.
55+
56+
The 1<sup>st</sup> point is addressed by the `check_for_pthread_cancel_race` call: yet another piece of per-thread state tracks whether a `pthread_cancel` call is in the middle of sending a signal, and if so, `check_for_pthread_cancel_race` waits for the signal delivery, thereby avoiding it from causing `EINTR` on an unrelated syscall.
57+
58+
The 2<sup>nd</sup> point is unaddressed. It is tracked as part of [glibc bug 12683](https://sourceware.org/bugzilla/show_bug.cgi?id=12683), where a solution has been proposed, but not yet implemented.
59+
60+
## Polyfilled semantics of cancellation points
61+
62+
For functions that don't meaningfully block in practice, such as `open_by_handle_at` and `getrandom`, the polyfill implementation of these functions inserts a `pthread_testcancel` call, as in:
63+
```c
64+
pthread_testcancel();
65+
return syscall(open_by_handle_at, ...);
66+
```
67+
68+
For functions that really can block, the polyfill implementation takes a two-pronged strategy:
69+
1. If the ambient glibc provides the function being polyfilled, call that.
70+
2. Otherwise, implement it similarly to how glibc currently does, albeit without the `check_for_pthread_cancel_race` call:
71+
72+
```c
73+
old_type = pthread_setcanceltype(ASYNCHRONOUS);
74+
result = syscall(epoll_pwait2, ...);
75+
pthread_setcanceltype(old_type);
76+
return result;
77+
```
78+
79+
If glibc one day fixes [bug 12683](https://sourceware.org/bugzilla/show_bug.cgi?id=12683), then using the ambient glibc implementation will give bug-free behaviour (when running with a sufficiently new glibc). On older glibc versions, the polyfill implementation is only marginally worse than the glibc implementation: the lack of a `check_for_pthread_cancel_race` call is unfortunate, but applications should be prepared to handle `EINTR` anyway.

0 commit comments

Comments
 (0)