Skip to content

TCG trace plugin #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 57 commits into
base: trace-10.0
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
9916241
Add skeleton implementation of tracing plugin.
Rot127 Jun 20, 2025
a0a98ed
Remove lockdown workflow
Rot127 Jun 20, 2025
dbafb0c
Move plugin into own dir.
Rot127 Jun 20, 2025
abf6040
Re-add tracewrap code from tracewrap-8.1-hexagon
Rot127 Jun 20, 2025
b96ce6a
Add preliminary README
Rot127 Jun 20, 2025
a040e41
Enable bap-tracing plugin build with meson.
Rot127 Jun 22, 2025
48e0f40
Add workflow to test the plugin build.
Rot127 Jun 22, 2025
e3a5dc8
Move defs into header file and add Frame struct.
Rot127 Jun 23, 2025
7054ef8
Imlpement plugin setup, vcpu init and locks for insn logging.
Rot127 Jun 23, 2025
b820d38
Init registers for VCPU
Rot127 Jun 23, 2025
531c8eb
Add mem access callback
Rot127 Jun 23, 2025
036ef5e
Rename buffer size constant
Rot127 Jun 23, 2025
4b3a908
Add operand append function for frames.
Rot127 Jun 23, 2025
e1f7bdc
Add an instruction object
Rot127 Jun 23, 2025
0c058f3
Implement update of register operands.
Rot127 Jun 23, 2025
dbbeb5c
Handle frame buffer flushing
Rot127 Jun 24, 2025
87407fe
Add pre register states and some clean up
Rot127 Jun 24, 2025
81a4ebc
Add bit length
Rot127 Jun 24, 2025
de922e9
Formatting
Rot127 Jun 24, 2025
8b0ae27
Add SPDX header
Rot127 Jun 24, 2025
8323fc7
Move and implement frame buffer functionality
Rot127 Jun 24, 2025
615f9bc
MOre implementation of frame functions.
Rot127 Jun 24, 2025
185c5bc
Move operand info generation to frame buffer
Rot127 Jun 24, 2025
451c231
Log the frame buffer array only as reader since each frame buffer is …
Rot127 Jun 24, 2025
ff88570
Implement frame buffer flush
Rot127 Jun 24, 2025
9d79ae1
Fix faulty name
Rot127 Jun 24, 2025
6c245ae
Add back some of the meta frame code
Rot127 Jun 24, 2025
8a20988
Add missing libprotobuf-c dependency
Rot127 Jun 24, 2025
c62b751
Several segfault fixes
Rot127 Jun 26, 2025
9d7f9aa
Add gdb register xml files.
Rot127 Jun 27, 2025
363aefe
Fix and print register tracing.
Rot127 Jun 27, 2025
f690488
Trace memory accesses
Rot127 Jun 27, 2025
cd45969
Write trace header (without machine details yet).
Rot127 Jun 27, 2025
09c5728
Docs add trace overview to README.
Rot127 Jun 28, 2025
4db5128
Use defined constant values of trace
Rot127 Jun 28, 2025
512a13a
Save TOC entries when writing frames
Rot127 Jun 28, 2025
c556a3f
Finish write of trace.
Rot127 Jun 28, 2025
4608ba4
Remove tracewrap files
Rot127 Jun 28, 2025
f88cb26
Write meta frames and add outputfile argument.
Rot127 Jun 30, 2025
00fd9c5
Document tracing
Rot127 Jun 30, 2025
799153c
Remove duplicate code.
Rot127 Jul 1, 2025
bc07dbf
Fix off by one
Rot127 Jul 1, 2025
d267206
Fix: Don't push invalid toc entry offset
Rot127 Jul 1, 2025
e91696e
Fix incorrect description of m in TOC index.
Rot127 Jul 2, 2025
44a49b6
Pad TOC entries when closing the trace.
Rot127 Jul 2, 2025
52e4c5a
Add mode of frame.
Rot127 Jul 2, 2025
f0525cb
Fix typo
Rot127 Jul 2, 2025
958dfd8
Fix too early return of partial mdoe match
Rot127 Jul 2, 2025
486c6b7
Add machine info for Sparc
Rot127 Jul 6, 2025
3ee7919
Fix register comparison for trace.
Rot127 Jul 7, 2025
cc17940
Remove padding frames again.
Rot127 Jul 7, 2025
6d61f9c
Store values always in little endian as the trace defines it.
Rot127 Jul 9, 2025
9ea6295
Formatting
Rot127 Jul 9, 2025
fe9340b
GDB stub should return data with endianess of the host machine.
Rot127 Jul 14, 2025
3dc6e18
Fix the endian issue once and for all.
Rot127 Jul 16, 2025
adb35d9
Use QEMU_PLUGIN_MEM_RW so also memory writes are detected.
Rot127 Jul 21, 2025
daa47ab
Fix typos, extend docs add tracing helper script
Rot127 Jul 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Build

on: [pull_request, workflow_dispatch]

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.x
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install deps
run: |
sudo apt-get -y update
sudo apt-get install -y git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ninja-build autoconf libtool protobuf-c-compiler libprotobuf-c-dev
- name: Install piqi
run: |
curl -OL https://raw.github.com/alavrik/piqi-binary/master/Linux-x86_64/piqi
chmod +x piqi
sudo mv piqi /usr/local/bin
piqi --version
- name: Checkout qemu
uses: actions/checkout@v4
with:
repository: BinaryAnalysisPlatform/qemu
path: qemu
submodules: true
- name: Build for Targets
run: |
cd qemu
mkdir build
cd build
../configure --enable-plugins --target-list=sparc-linux-user,sparc64-linux-user
ninja
30 changes: 0 additions & 30 deletions .github/workflows/lockdown.yml

This file was deleted.

3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,6 @@
[submodule "tests/lcitool/libvirt-ci"]
path = tests/lcitool/libvirt-ci
url = https://gitlab.com/libvirt/libvirt-ci.git
[submodule "contrib/plugins/bap-tracing/bap-frames"]
path = contrib/plugins/bap-tracing/bap-frames
url = [email protected]:BinaryAnalysisPlatform/bap-frames.git
104 changes: 104 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# BAP emulation trace generator

This QEMU fork implements the TCG plugin to generate execution traces in the
[bap-frame](https://github.com/BinaryAnalysisPlatform/bap-frames) format.

This plugin does not yet support all targets.
If not listed below it is untested.

Known to work:

- Sparc
- Hexagon
- PPC

Needs fixes:

- ARM (cannot get current mode of VCPU if target can switch between ARM/Thumb).

Previous traces were generated with a patched QEMU.
You can find these in tracewrap-* branches.

## Dependencies

1. Install [piqi](https://piqi.org/downloads/) so you have the `piqi` binary in `PATH`.
2. Install the developer package of `protobuf-c`. E.g. `protobuf-c-devel` (Fedora), `libprotobuf-c-dev` (Debian).
3. QEMU dependencies (see [QEMU docs](https://www.qemu.org/docs/master/devel/build-environment.html)).

## Building

```bash
mkdir build
cd build
# See `../configure --help` for a list of targets.
../configure --enable-plugins --target-list=<target>
make
```

## Tracing a binary

The plugin takes two required arguments:

`bin_path`: The path to the binary emulated. Due to a [QEMU bug](https://gitlab.com/qemu-project/qemu/-/issues/3014) this cannot be inferred.
`out`: The output file to save the trace into.
`endianness`: The architecture endanness.

```bash
./qemu-sparc64 -plugin file=buil/contrib/plugins/bap-tracing/libbap_tracing.so,bin_path=<bin_path>,out=<output-file>,endianness=[b/l] -d plugin <bin_path>
ls <output-file>
```

You can also use the helper shell script:

```bash
./gen-trace.sh ./build/ sparc64 b <path_to_bin>
```

> [!NOTE]
> The trace plugin currently only generates standard frames.
> This is due to the limitations of the QEMU plugin API.
>
> If the traced binary exits due to an exception it can only indirectly be observed.
> It will produce a standard frame without any logged post register state.
> Any completed memory read/write might still be logged.
>
> If you suspect this, execute the binary with the `execlog` plugin (see `gen-trace.sh`)
> to check of the execution stops earlier than expected.


## Trace format

The generated trace consists of three parts: the header,
a table of contents (TOC) holding the frame entries, and an index into the TOC.

Each frame entry starts with the size of the frame, followed by the actual frame data.
A fixed number of frame entries are considered one _entry_ in the TOC.

The TOC index is stored at the end.

For specifics about the frame contents, please refer
to the [definitions](https://github.com/BinaryAnalysisPlatform/bap-frames/tree/master/piqi) in
the BAP-frames repository.

**Format**

| Offset | Type | Field | Trace section |
|--------|------|-------|------|
| 0x0 | uint64_t | magic number (7456879624156307493LL) | Header begin |
| 0x8 | uint64_t | trace version number | |
| 0x10 | uint64_t | frame_architecture | |
| 0x18 | uint64_t | frame_machine, 0 for unspecified. | |
| 0x20 | uint64_t | n = total number of frames in trace. | |
| 0x28 | uint64_t | T = offset to TOC index. | |
| 0x30 | uint64_t | sizeof(frame_0) | TOC begin |
| 0x38 | meta_frame | frame_0 | |
| 0x40 | uint64_t | sizeof(frame_1) | |
| 0x48 | type(frame_1) | frame_1 | |
| ... | ... | ... | |
| T-0x10 | uint64_t | sizeof(frame_n-1) | |
| T-0x8 | type(frame_n-1) | frame_n-1 | |
| T+0 | uint64_t | m = number of frames per TOC entry | TOC index begin |
| T+0x8 | uint64_t | offset toc_entry(0) | |
| T+0x10 | uint64_t | offset toc_entry(1) | |
| ... | ... | ... | |
| T+0x8+(0x8*ceil(n/m)) | uint64_t | offset toc_entry(ceil(n/m)) | |
1 change: 1 addition & 0 deletions configs/targets/sparc-linux-user.mak
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ TARGET_ARCH=sparc
TARGET_SYSTBL_ABI=common,32
TARGET_SYSTBL=syscall.tbl
TARGET_BIG_ENDIAN=y
TARGET_XML_FILES= gdb-xml/sparc32-core.xml gdb-xml/sparc32-cp0.xml gdb-xml/sparc32-cpu.xml gdb-xml/sparc32-fpu.xml
TARGET_LONG_BITS=32
1 change: 1 addition & 0 deletions configs/targets/sparc-softmmu.mak
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
TARGET_ARCH=sparc
TARGET_BIG_ENDIAN=y
TARGET_SUPPORTS_MTTCG=y
TARGET_XML_FILES= gdb-xml/sparc32-core.xml gdb-xml/sparc32-cp0.xml gdb-xml/sparc32-cpu.xml gdb-xml/sparc32-fpu.xml
TARGET_LONG_BITS=32
1 change: 1 addition & 0 deletions configs/targets/sparc32plus-linux-user.mak
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ TARGET_ABI_DIR=sparc
TARGET_SYSTBL_ABI=common,32
TARGET_SYSTBL=syscall.tbl
TARGET_BIG_ENDIAN=y
TARGET_XML_FILES= gdb-xml/sparc64-core.xml gdb-xml/sparc64-cp0.xml gdb-xml/sparc64-cpu.xml gdb-xml/sparc64-fpu.xml
TARGET_LONG_BITS=64
1 change: 1 addition & 0 deletions configs/targets/sparc64-linux-user.mak
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@ TARGET_ABI_DIR=sparc
TARGET_SYSTBL_ABI=common,64
TARGET_SYSTBL=syscall.tbl
TARGET_BIG_ENDIAN=y
TARGET_XML_FILES= gdb-xml/sparc64-core.xml gdb-xml/sparc64-cp0.xml gdb-xml/sparc64-cpu.xml gdb-xml/sparc64-fpu.xml
TARGET_LONG_BITS=64
1 change: 1 addition & 0 deletions configs/targets/sparc64-softmmu.mak
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ TARGET_ARCH=sparc64
TARGET_BASE_ARCH=sparc
TARGET_BIG_ENDIAN=y
TARGET_SUPPORTS_MTTCG=y
TARGET_XML_FILES= gdb-xml/sparc64-core.xml gdb-xml/sparc64-cp0.xml gdb-xml/sparc64-cpu.xml gdb-xml/sparc64-fpu.xml
TARGET_LONG_BITS=64
1 change: 1 addition & 0 deletions contrib/plugins/bap-tracing/bap-frames
Submodule bap-frames added at 713681
22 changes: 22 additions & 0 deletions contrib/plugins/bap-tracing/fix_proto_src.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/env python3
"""
This just does:
sed -i 's/->base/->__base/g' <file>
sed -i 's/ProtobufCMessage base;/ProtobufCMessage __base;/g' <file>
"""

import sys
import re

if len(sys.argv) != 6 or sys.argv[3] != "-o":
print("usage: fix_proto_src.py frame.piqi.pb-c.c frame.piqi.pb-c.h -o frame.piqi.pb-c-fixed.c frame.piqi.pb-c-fixed.h")
exit(1)

for (in_file, out_file) in zip(sys.argv[1:3], sys.argv[4:6]):
with open(in_file, "r") as i:
contents = i.read()
contents = contents.replace("->base", "->__base")
contents = contents.replace("ProtobufCMessage base;", "ProtobufCMessage __base;")
contents = re.sub(r'".*frame.piqi.pb-c.h"', '"frame.piqi.pb-c-patched.h"', contents)
with open(out_file, "w") as o:
o.write(contents)
Loading
Loading