-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathasmdiff.py
More file actions
executable file
·1963 lines (1748 loc) · 82.8 KB
/
Copy pathasmdiff.py
File metadata and controls
executable file
·1963 lines (1748 loc) · 82.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
#!/usr/bin/env python3
"""Per-function assembly inspection and comparison, across a compiler matrix or from a shipped ELF.
Compiles a C source file across a matrix of compilers, extracts each
variant function's assembly from the -S output, and prints side-by-side
listings plus a summary of instruction counts, loop spans (instructions
between a local label and its last backward branch), and outbound calls.
Automates fold-vs-libcall analysis when evaluating micro-optimisations
(e.g. "does this still compile to one instruction, or is it a libcall?").
Usage:
tools/asmdiff/asmdiff.py SOURCE.c [SOURCE2.c | FUNC...]
[--pair OLD:NEW]... [--across FUNC]...
[--cc 'CC FLAGS']... [--target NAME]...
[--config PATH] [--compile-commands [PATH]]
[--flags-like PATH] [--db-includes]
[--summary-only] [--collapse] [--span-stats]
[--json] [--layout list|side-by-side] [-v]
[-- EXTRA_FLAGS...]
tools/asmdiff/asmdiff.py FIRMWARE.elf [FUNC...] [--filter REGEX]
[--objdump PATH] [--summary-only]
[--span-stats] [--json]
tools/asmdiff/asmdiff.py --edit-config | --example-config | --version
Five modes:
SOURCE.c FUNC inspect: print the named function's assembly, no
comparison. One usable compiler prints a listing
plus a stats row, exactly two print side by side,
more print one block per compiler; --layout forces
list or side-by-side.
FIRMWARE.elf ELF input (detected by magic bytes): disassemble a
linked binary through the toolchain's objdump and
run named and/or --filter REGEX matched functions
through the same analyzers - post-LTO/link reality
instead of single-TU codegen. objdump comes from
--objdump PATH or the first gcc in the matrix
(trailing gcc swapped for objdump); Xtensa
-mlongcalls call sequences are resolved to their
real callees when the disassembly names them.
--pair OLD:NEW compares two different functions within one compilation
(with no --pair, old_X/new_X names auto-pair).
--across FUNC compares the SAME function across two compilations:
either one file under two --cc entries (flag/define
variants), or two source files (before/after versions)
under each compiler in the matrix.
(neither) whole-file summary: per-function counts plus a file
total, for one file or side by side for two.
Four flags shape the output: --summary-only prints only the stats
tables, --collapse elides identical runs in side-by-side listings
keeping context around each difference, --span-stats follows each
stats table with a per-loop-span instruction mix (load/store/mul/
branch/other counts; branch includes calls), and --json replaces the
tables with one JSON document of per-function records (implies
--summary-only) for scripted callers.
Compilers come from --cc strings, from named targets in an asmdiff.toml
config file (--target NAME), from the config's `default` entry, or —
failing all of those — plain `gcc -O3` and `clang -O3`.
Flags after a bare `--` are appended to every compiler invocation.
--edit-config opens the config (--config PATH, else ~/.config/
asmdiff.toml) in $VISUAL/$EDITOR, creating it from the built-in example
when missing; --example-config prints that example to stdout.
A config target may name a compile_commands.json (compile_commands = PATH):
the include/define flags recorded there for the source being compiled are
added to that target's command, so a real project source resolves its
headers the way its own build system does (e.g. an ESP-IDF component).
compile_commands = true in a target — or a bare --compile-commands on the
command line — instead finds the database by checking each directory from
the CWD up to the repository root, and its build/ subdirectory; a source
absent from a database found that way is compiled without borrowed flags
(with a note) rather than being an error. --flags-like PATH lets such a
source borrow the flags recorded for PATH, so a modified copy compares
against its original under one header environment. --db-includes limits
any borrow to the header-search paths, re-emitted as -idirafter so they
cannot shadow the host's own system headers: a host target can then
compile a cross project's source without inheriting cross-only defines,
-specs, or a libc-overlay include directory.
"""
import argparse
import difflib
import glob
import hashlib
import json
import os
import re
import shlex
import shutil
import subprocess
import sys
import tempfile
from itertools import zip_longest
from pathlib import Path
try:
import tomllib # Python >= 3.11; config files are optional without it
except ModuleNotFoundError:
tomllib = None
# Single source of truth for the package version: pyproject.toml reads
# it from here (hatchling dynamic version).
__version__ = "0.3.0"
DEFAULT_COMPILERS = ["gcc", "clang"]
FALLBACK_FLAGS = "-O3"
CONFIG_NAME = "asmdiff.toml"
# Mirror of asmdiff.example.toml, embedded because the wheel ships only
# this module: --example-config prints it and --edit-config seeds a new
# config from it. A unit test keeps it byte-identical to the repo file.
EXAMPLE_CONFIG = """\
# asmdiff config — named compiler+flags targets.
#
# Copy to one of the locations asmdiff searches (first hit wins):
# 1. the path given with --config
# 2. asmdiff.toml next to the SOURCE.c being compiled
# 3. asmdiff.toml in the current directory
# 4. ~/.config/asmdiff.toml
#
# Each [table] is a target usable as `--target NAME`; the optional
# top-level `default` names the target(s) used when no --cc/--target
# is given (a list runs several: default = ["gcc", "clang"]).
# Compile at the flags your project ships with — that is the whole
# point of the tool.
default = "gcc"
[gcc]
cc = "gcc"
flags = [
"-O3", "-Wall", "-Wextra",
# Project-specific include paths and defines go here:
# "-I/path/to/project/src", "-DMY_FEATURE",
]
[clang]
cc = "clang"
flags = ["-O3", "-Wall", "-Wextra"]
# The name the docs use for "your native compiler" (--target host): an
# alias of [gcc] kept as its own table so the examples work verbatim.
[host]
cc = "gcc"
flags = ["-O3", "-Wall", "-Wextra"]
# cc values may use ~, $VARS, and glob patterns. A glob that matches
# several installed toolchains resolves to the highest version-sorted
# one (announced on stderr); pin the exact esp-NN directory instead if
# reproducibility across sessions matters more than convenience.
# On native Windows HOME is usually unset - use $USERPROFILE (or
# %USERPROFILE%) in the patterns below instead.
#
# The ESP profiles below keep flags minimal (-O2 plus arch selection);
# append what your project ships with (-DNDEBUG, -Os, -I..., ...).
# A source that pulls in framework headers (ESP-IDF's freertos/*, a
# generated sdkconfig.h, ...) additionally needs the build's -I/-D
# flags: add `compile_commands = true` (or a path) to any target to
# borrow them from the build's compile_commands.json - see README.
# --- Profile: riscv32-esp (ESP32-C3 / C6 / H2 / P4) -------------------
# All RISC-V ESP chips share one riscv32-esp-elf-gcc binary; the
# targets differ only in -march/-mabi. Uncomment to run the whole
# profile as the default matrix:
# default = ["esp32c3", "esp32c6", "esp32h2", "esp32p4"]
[esp32c3]
cc = "$HOME/.espressif/tools/riscv32-esp-elf/esp-*/riscv32-esp-elf/bin/riscv32-esp-elf-gcc"
flags = ["-O2", "-march=rv32imc_zicsr_zifencei", "-mabi=ilp32"]
[esp32c6]
cc = "$HOME/.espressif/tools/riscv32-esp-elf/esp-*/riscv32-esp-elf/bin/riscv32-esp-elf-gcc"
flags = ["-O2", "-march=rv32imac_zicsr_zifencei", "-mabi=ilp32"]
[esp32h2]
cc = "$HOME/.espressif/tools/riscv32-esp-elf/esp-*/riscv32-esp-elf/bin/riscv32-esp-elf-gcc"
flags = ["-O2", "-march=rv32imac_zicsr_zifencei", "-mabi=ilp32"]
# ESP32-P4 is the only ESP RISC-V chip with an FPU, hence the
# hard-float ABI.
[esp32p4]
cc = "$HOME/.espressif/tools/riscv32-esp-elf/esp-*/riscv32-esp-elf/bin/riscv32-esp-elf-gcc"
flags = ["-O2", "-march=rv32imafc_zicsr_zifencei", "-mabi=ilp32f"]
# --- Profile: xtensa-esp (ESP32 / S2 / S3) ----------------------------
# The unified xtensa-esp-elf toolchain ships one gcc binary per chip.
# default = ["esp32", "esp32s2", "esp32s3"]
[esp32]
cc = "$HOME/.espressif/tools/xtensa-esp-elf/esp-*/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc"
flags = ["-O2", "-mlongcalls"]
[esp32s2]
cc = "$HOME/.espressif/tools/xtensa-esp-elf/esp-*/xtensa-esp-elf/bin/xtensa-esp32s2-elf-gcc"
flags = ["-O2", "-mlongcalls"]
[esp32s3]
cc = "$HOME/.espressif/tools/xtensa-esp-elf/esp-*/xtensa-esp-elf/bin/xtensa-esp32s3-elf-gcc"
flags = ["-O2", "-mlongcalls"]
# --- Profile: other embedded (STM32 / RP2350) -------------------------
# Unlike the ESP toolchains these have no single well-known install
# path: the compilers must be on PATH, or edit cc to a full path.
# Any Cortex-M STM32; adjust -mcpu to your family (cortex-m0plus,
# cortex-m3, cortex-m7, cortex-m33, ...) and add -mfloat-abi/-mfpu
# flags if your project uses the FPU.
[stm32]
cc = "arm-none-eabi-gcc"
flags = ["-O2", "-mcpu=cortex-m4", "-mthumb"]
# RP2350 Hazard3 cores in RISC-V mode; riscv64-unknown-elf-gcc is
# multilib, -march/-mabi select rv32. Hazard3 also implements the
# Zba/Zbb/Zbs/Zbkb extensions (append to -march on gcc >= 13, as
# pico-sdk does).
[rp2350]
cc = "riscv64-unknown-elf-gcc"
flags = ["-O2", "-march=rv32imac_zicsr_zifencei", "-mabi=ilp32"]
"""
# Preprocessor flags lifted from a compile_commands.json entry so a project
# source compiles the way its build system compiles it: header search paths,
# forced includes, and defines. Everything else the entry records (the
# compiler, -O/-std/-W flags, -c, -o OUT, the source itself) is ignored —
# asmdiff supplies the compiler and optimisation flags from the target.
# Path-bearing flags may be glued (-Ipath) or split (-I path); defines too
# (-DFOO / -D FOO). None of the path flags is a prefix of another and -I is
# case-distinct from -i*, so a single left-to-right scan is unambiguous.
_CC_DB_PATH_FLAGS = ("-I", "-iquote", "-isystem", "-idirafter",
"-include", "-imacros", "-isysroot")
_CC_DB_PLAIN_FLAGS = ("-D", "-U")
# Driver-level flags that also shape the header environment: a specs file
# can swap the entire libc header set (ESP-IDF v6 selects picolibc via
# -specs=picolibc.specs), and --sysroot moves every system include. Both
# accept =-glued and split spellings; both are re-emitted =-glued.
_CC_DB_EQ_FLAGS = ("-specs", "--specs", "--sysroot")
_DB_CACHE = {}
_MISS_NOTED = set()
_BORROW_NOTED = set()
# --db-includes: borrow only the header-search paths from the database.
# A project's -I paths are valid on any target; its -D defines, forced
# includes, and -specs/--sysroot are tied to the arch the database was
# built for — this is how a host target compiles a cross project's
# source without inheriting cross-only flags.
DB_INCLUDES = False
_DB_SEARCH_PATH_FLAGS = ("-I", "-iquote", "-isystem", "-idirafter")
# --flags-like PATH: a source with no database entry borrows the flags
# recorded for PATH. This is how a modified copy of a project source gets
# the same header environment as its original, so an --across between them
# compares codegen instead of header configurations.
FLAGS_LIKE = None
class Target(str):
"""A compiler-matrix entry: the ``CC FLAGS`` command string, plus an
optional compile_commands.json whose per-source include/define flags are
appended at compile time. Subclassing str means it prints, compares, and
shlex-splits as the bare command everywhere the matrix is consumed, so
only compile_to_asm needs to know about the extra attributes.
``db_discovered`` marks a database that was found by searching near the
CWD rather than named explicitly; a source absent from a discovered
database is tolerated (compiled without borrowed flags, with a note),
where an explicit database makes that an error."""
def __new__(cls, cmd, compile_commands=None, db_discovered=False):
self = super().__new__(cls, cmd)
self.compile_commands = compile_commands
self.db_discovered = db_discovered
return self
def _abs_against(directory, value):
"""Resolve a path recorded in a compile_commands entry against that
entry's ``directory``, so a relative -I still works from asmdiff's CWD."""
p = Path(value)
if directory and not p.is_absolute():
p = Path(directory) / p
return str(p)
def _expand_response_files(tokens, directory, depth=0):
"""Inline GCC @file response files: each @FILE token is replaced by the
shlex-split contents of FILE, resolved against the entry's ``directory``
(the CWD the driver ran from). Build systems park header-environment
flags there — ESP-IDF v6 hides -specs=picolibc.specs in one — so
skipping them silently loses flags this feature exists to borrow.
An unreadable file is warned about and dropped; nesting is bounded as
a cycle guard.
"""
out = []
for tok in tokens:
if not tok.startswith("@") or len(tok) == 1:
out.append(tok)
continue
path = _abs_against(directory, tok[1:])
try:
content = Path(path).read_text()
except OSError:
print(f"warning: response file {path} not readable; "
"flags inside it are not borrowed", file=sys.stderr)
continue
inner = shlex.split(content)
if depth < 8:
inner = _expand_response_files(inner, directory, depth + 1)
out += inner
return out
def _specs_value(directory, value):
"""A specs argument with a path separator is a file path (resolved like
any other); a bare name (picolibc.specs) is looked up in the compiler's
own search directories and must pass through untouched."""
if "/" in value or os.sep in value:
return _abs_against(directory, value)
return value
def include_flags(tokens, directory):
"""Pick header-search, forced-include, and define flags out of one
recorded compile command; make relative paths absolute against
``directory``. Glued and split spellings are both recognised; path
flags are re-emitted in split form (-I path), which every driver accepts.
"""
flags = []
i, n = 0, len(tokens)
while i < n:
tok, extra, matched = tokens[i], 0, False
for f in _CC_DB_PATH_FLAGS:
if tok == f and i + 1 < n: # -I path
flags += [f, _abs_against(directory, tokens[i + 1])]
extra, matched = 1, True
break
if tok.startswith(f) and len(tok) > len(f): # -Ipath
flags += [f, _abs_against(directory, tok[len(f):])]
matched = True
break
if not matched:
for f in _CC_DB_EQ_FLAGS:
value = None
if tok == f and i + 1 < n: # -specs file
value, extra = tokens[i + 1], 1
elif tok.startswith(f + "="): # -specs=file
value = tok[len(f) + 1:]
if value is not None:
resolve = (_specs_value if f.endswith("specs")
else _abs_against)
flags.append(f + "=" + resolve(directory, value))
matched = True
break
if not matched:
for f in _CC_DB_PLAIN_FLAGS:
if tok == f and i + 1 < n: # -D FOO
flags.append(f + tokens[i + 1])
extra = 1
break
if tok.startswith(f) and len(tok) > len(f): # -DFOO
flags.append(tok)
break
i += 1 + extra
return flags
def find_compile_commands():
"""Locate a compile_commands.json by walking up from the current
directory.
Each directory from the CWD upward is checked for the database itself,
then for build/compile_commands.json (where CMake and idf.py leave it),
so the search works from a project root or from any depth of component
directory. The walk stops at the first directory containing .git (the
repository root — checked after that directory's own candidates) or
after a bounded number of levels, so an unrelated database further up
the filesystem is never picked up. Returns None when nothing is found.
"""
d = Path.cwd()
for _ in range(10):
for candidate in (d / "compile_commands.json",
d / "build" / "compile_commands.json"):
if candidate.is_file():
return str(candidate)
if (d / ".git").exists() or d.parent == d:
return None
d = d.parent
return None
def _discovered_db(who):
"""find_compile_commands() for a caller that opted in (compile_commands
= true, or a bare --compile-commands): finding nothing is then an error,
not a silent no-op."""
found = find_compile_commands()
if found is None:
sys.exit(f"error: {who}: no compile_commands.json found in the "
"current directory, its build/, or any parent up to the "
"repository root")
return found
def load_compile_commands(path):
"""Parse a compile_commands.json into its list of entries (cached, since
one database is queried once per source per compiler in the matrix)."""
if path in _DB_CACHE:
return _DB_CACHE[path]
try:
data = json.loads(Path(path).read_text())
except FileNotFoundError:
sys.exit(f"error: compile_commands.json not found: {path}")
except (json.JSONDecodeError, OSError) as exc:
sys.exit(f"error: {path}: {exc}")
if not isinstance(data, list):
sys.exit(f"error: {path}: expected a JSON array of compile entries")
_DB_CACHE[path] = data
return data
def _borrowed(flags):
"""Apply --db-includes to one borrowed flag list: keep the
header-search pairs (-I/-iquote/-isystem/-idirafter PATH), drop
defines, forced includes, and driver-level flags. Identity when
the option is off.
Kept paths are re-emitted as -idirafter — searched AFTER the
compiler's own system directories — because a cross project's
include set can overlay libc headers (ESP-IDF ships an esp_libc/
platform_include whose stdio.h would otherwise shadow the host's);
project-only headers still resolve, the host's libc wins.
"""
if not DB_INCLUDES:
return flags
kept, i = [], 0
while i < len(flags):
f = flags[i]
if f in _CC_DB_PATH_FLAGS: # split "-I path" pair
if f in _DB_SEARCH_PATH_FLAGS:
kept += ["-idirafter", flags[i + 1]]
i += 2
else: # -DFOO / -specs=... token
i += 1
return kept
def compile_commands_flags(db_path, source, missing_ok=False):
"""Header/define flags for ``source`` taken from a compile_commands.json.
The entry whose ``file`` resolves to the same path as ``source`` supplies
its include/define flags (relative paths made absolute against the entry's
``directory``). A ``command`` string is tokenised; an ``arguments`` array
is used as-is. A source absent from the database is an error — compiling
it would otherwise fail on the very header this feature exists to supply —
and the message flags a same-name entry recorded under a different path.
With ``missing_ok`` (auto-discovered databases) an absent source instead
compiles without borrowed flags, after a one-time note per (db, source).
In either mode, --flags-like PATH satisfies an absent source with the
flags recorded for PATH — the modified-copy workflow.
"""
entries = load_compile_commands(db_path)
want = Path(source).resolve()
flags, name_seen = _lookup_entry(entries, want)
if flags is not None:
return _borrowed(flags)
if FLAGS_LIKE:
like = Path(FLAGS_LIKE).resolve()
like_flags, _ = _lookup_entry(entries, like)
if like_flags is None:
sys.exit(f"error: --flags-like {FLAGS_LIKE} "
f"not found in {db_path}")
key = (db_path, str(want))
if key not in _BORROW_NOTED:
_BORROW_NOTED.add(key)
print(f"note: {source} not in {db_path}; borrowing the flags "
f"recorded for {FLAGS_LIKE}", file=sys.stderr)
return _borrowed(like_flags)
suggest = ("; --flags-like PATH can borrow another entry's flags"
if name_seen else "")
if missing_ok:
key = (db_path, str(want))
if key not in _MISS_NOTED:
_MISS_NOTED.add(key)
print(f"note: {source} not in {db_path}; compiling without "
f"its include/define flags{suggest}", file=sys.stderr)
return []
hint = (f"; an entry named {want.name} exists under a different path — "
"pass the source as it appears in the database, or borrow its "
"flags with --flags-like") if name_seen else ""
sys.exit(f"error: {source} not found in {db_path}{hint}")
def _lookup_entry(entries, want):
"""Flags for the entry whose file resolves to ``want``, or None.
The second result reports whether some entry shares ``want``'s
basename — the raw material for the miss hints above.
"""
name_seen = False
for entry in entries:
f = entry.get("file")
if not f:
continue
directory = entry.get("directory", "")
fp = Path(f)
if not fp.is_absolute():
fp = Path(directory) / fp
try:
same = fp.resolve() == want
except OSError:
same = False
if same:
args = entry.get("arguments")
tokens = (list(args) if isinstance(args, list)
else shlex.split(entry.get("command", "")))
tokens = _expand_response_files(tokens, directory)
return include_flags(tokens, directory), name_seen
if Path(f).name == want.name:
name_seen = True
return None, name_seen
# A label at column 0 that is not a local (.L*) label starts a function.
FUNC_LABEL = re.compile(r"^([A-Za-z_][\w$.]*):")
# .type NAME, @object|@function — gcc/clang emit this before the label on
# ELF. Objects (string constants like __func__, global state, LUTs) get
# column-0 labels too and must not be reported as functions. The type
# marker is @ on x86/Xtensa/RISC-V, % on ARM, # on some targets.
TYPE_DIRECTIVE = re.compile(r'^\s*\.type\s+([^,\s]+)\s*,\s*[@%#]?(\w+)')
# Assembler directives that carry no information worth reading. .local/
# .comm/.lcomm declare zero-initialised data, .literal/.literal_position
# are Xtensa literal-pool bookkeeping — none of them is an instruction.
NOISE = re.compile(
r"^\s*\.(cfi_|p2align|align\b|loc\b|file\b|text\b|globl\b|global\b|"
r"type\b|section\b|ident\b|weak\b|hidden\b|addrsig|build_version|"
r"local\b|comm\b|lcomm\b|literal_position|literal\b)"
)
# Compiler-generated bracketing labels that add nothing (.LFB0:, .Lfunc_end0:).
NOISE_LABEL = re.compile(r"^\.(LFB|LFE|Lfunc_begin|Lfunc_end)\d*:")
# Data emitted *inside* a function body: switch jump tables (.long/.word
# entries), inline constants, strings. These are not instructions, so they
# must not be counted; and a self-relative table entry (".long .L5-.L4")
# references its base label from below, which the loop-span scan would
# otherwise read as a backward branch and report as a phantom loop.
DATA = re.compile(
r"^\.(long|quad|word|hword|short|byte|[248]byte|value|zero|octa|"
r"string|ascii|asciz|single|double|float|dc(\.[abwlq])?)\b"
)
def extract_functions(asm_text):
"""Map function name -> cleaned asm lines from compiler -S output.
A function body runs from its column-0 label to the matching .size
directive (gcc and clang both emit one on ELF) or the next function
label. Labels typed ``@object`` (string constants, global state,
lookup tables) are data, not functions, and are skipped entirely;
an untyped label still counts as a function so hand-written asm
keeps working. Comment lines, CFI/section/alignment directives,
compiler bracketing labels, and inline data (switch jump tables,
constants) are dropped; instructions and meaningful local labels
(loop targets) are kept, whitespace-stripped.
"""
funcs = {}
current = None
data_labels = set()
for raw in asm_text.splitlines():
t = TYPE_DIRECTIVE.match(raw)
if t and t.group(2) != "function":
data_labels.add(t.group(1))
m = FUNC_LABEL.match(raw)
if m:
if m.group(1) in data_labels:
current = None
else:
current = m.group(1)
funcs[current] = []
continue
if current is None:
continue
if re.match(r"^\s*\.size\b", raw):
current = None
continue
line = raw.strip()
if not line or line.startswith(("#", "//")):
continue
if NOISE.match(line) or NOISE_LABEL.match(line) or DATA.match(line):
continue
funcs[current].append(line)
return funcs
# `objdump -d` function header: "40370400 <render_lut>:".
OBJDUMP_HEADER = re.compile(r"^([0-9a-f]+) <([^>]+)>:\s*$")
# `objdump -d` instruction line: address, one byte-dump field (hex digits
# and spaces; "004136" is the encoding, not the mnemonic), then the
# mnemonic and operands.
OBJDUMP_INSN = re.compile(
r"^\s*([0-9a-f]+):\t[0-9a-f ]+\t\s*(\S+)[ \t]*(.*?)\s*$")
# An "addr <annotation>" operand: branch, loop or call target.
OBJDUMP_TARGET = re.compile(r"\b([0-9a-f]+) <([^>]+)>")
def extract_functions_objdump(dump_text):
"""Map function name -> cleaned asm lines from `objdump -d` output.
Produces the shape extract_functions() yields for -S output, so
analyze() and loop_spans() work unchanged: one mnemonic + operands
per line, labels ending with ':'. A linked ELF has addresses where
-S output has labels, so in-function branch targets are rewritten to
synthetic local labels (.L<hex-offset>) inserted at the target
instruction; an Xtensa zero-overhead loop's end address becomes
.L<off>_LEND, keeping the ZOL-vs-branch naming convention -S output
has. Targets outside the function keep their symbol annotation, so
calls still name their callee. Offset-form headers
(``<sym+0x..>``/``<sym-0x..>``: literal pools and other symbol-less
gaps that disassemble as garbage) and ``...`` filler are skipped.
"""
funcs = {}
current = None
for raw in dump_text.splitlines():
h = OBJDUMP_HEADER.match(raw)
if h:
name = h.group(2)
if re.search(r"[+-]0x[0-9a-f]+$", name):
current = None # symbol-less gap, not a function
else:
current = []
funcs[name] = (int(h.group(1), 16), current)
continue
if current is None:
continue
m = OBJDUMP_INSN.match(raw)
if m:
current.append((int(m.group(1), 16), m.group(2), m.group(3)))
return {name: _relabel_objdump(start, _resolve_longcalls(insns))
for name, (start, insns) in funcs.items()}
# The value annotation objdump appends to an l32r whose literal resolves
# to a bare symbol: "l32r a8, <lit> (40002274 <__divsf3>)". An offset
# form (<_etext+0x100>) is a constant that happens to fall near a
# symbol, not a callee, so it deliberately does not match.
L32R_VALUE = re.compile(r"\([0-9a-f]+ <([A-Za-z_][\w$.]*)>\)$")
# Xtensa integer stores: the only mnemonics whose first a-register
# operand is read, not written. Everything else writing its first
# operand (l32i, mov.n, arithmetic) clobbers a loaded call target.
XTENSA_STORES = {"s8i", "s16i", "s32i", "s32i.n", "s32e"}
def _resolve_longcalls(insns):
"""Name the callees of Xtensa -mlongcalls sequences.
-mlongcalls emits every call as "l32r aN, <lit>" + "callx8 aN"; the
linker relaxes in-range pairs back to call8, so the ones left in an
ELF are real out-of-range calls (typically flash to ROM). When the
l32r's value annotation names a symbol, rewrite the callx operand to
it so CALL_RE reports the callee. The nearest write to the call's
register within the previous 8 instructions decides: an annotated
l32r resolves, anything else (an unannotated l32r, or a non-store
overwriting the register - e.g. l32i fetching a function pointer
out of a just-loaded struct) leaves the call indirect. A wrong
callee would be worse than a register name.
"""
out = list(insns)
for i, (addr, mnem, ops) in enumerate(insns):
if not (re.fullmatch(r"callx\d+", mnem)
and re.fullmatch(r"a\d+", ops)):
continue
for j in range(i - 1, max(i - 9, -1), -1):
_, mid_mnem, mid_ops = insns[j]
if not mid_ops.startswith(ops + ","):
continue
if mid_mnem == "l32r":
value = L32R_VALUE.search(mid_ops)
if value:
out[i] = (addr, mnem, value.group(1))
break
if mid_mnem not in XTENSA_STORES:
break # register rewritten in between
return out
def _relabel_objdump(start, insns):
"""Turn one function's (addr, mnemonic, operands) rows into cleaned
lines, synthesizing local labels for in-function targets."""
addrs = {addr for addr, _, _ in insns}
labels = {}
for _, mnem, ops in insns:
for m in OBJDUMP_TARGET.finditer(ops):
taddr = int(m.group(1), 16)
if taddr not in addrs:
continue
if mnem.startswith("loop"): # ZOL references its END
labels[taddr] = ".L%x_LEND" % (taddr - start)
elif taddr not in labels:
labels[taddr] = ".L%x" % (taddr - start)
def target_name(m):
return labels.get(int(m.group(1), 16), m.group(2))
lines = []
for addr, mnem, ops in insns:
if addr in labels:
lines.append(labels[addr] + ":")
ops = OBJDUMP_TARGET.sub(target_name, ops)
lines.append(mnem + "\t" + ops if ops else mnem)
return lines
# Direct-call / tail-call mnemonics across x86 (call, jmp), ARM (bl, blx),
# RISC-V (call, tail, jal), and Xtensa (call0/4/8/12, callx*, j). Longest
# alternatives first so e.g. "callx8" is not consumed as "call". The
# symbol must be the sole/final operand (optionally @PLT-suffixed), so
# multi-operand forms like "jal ra, exp2f" don't report the register.
CALL_RE = re.compile(
r"^(?:callx\d+|call\d*|callq|jalr|jal|jmp|blx|bl|tail|j)\s+"
r"([A-Za-z_][\w$.]*)(?:@[\w.]+)?\s*(?:[#;].*)?$"
)
def analyze(lines):
"""Return (instruction_count, called_symbols) for cleaned asm lines.
A call is a call/tail-call mnemonic whose first operand looks like a
symbol name — local labels (.L*) and %-registers never match, so
branches inside the function are not counted.
"""
insns = 0
calls = []
for line in lines:
if line.endswith(":"):
continue
insns += 1
m = CALL_RE.match(line)
if m:
sym = m.group(1)
if sym not in calls:
calls.append(sym)
return insns, calls
# A local-label operand (branch target, zero-overhead loop end). Literal
# pool labels (.LC0) also match, but they are emitted outside function
# bodies, so they never appear in the label map built from a body.
LABEL_REF = re.compile(r"\.L[\w$.]+")
def loop_span_ranges(lines):
"""Return [(label, lo, hi)] line-index ranges for loop spans.
A span is the run of instructions from a local label to the last
instruction that references it from below — a backward branch, which
is what a compiled loop looks like on every target the tool parses.
Xtensa zero-overhead loops (loop/loopnez/loopgt) reference their END
label instead; there the span is the instructions the loop encloses.
Spans are reported in order of appearance, one per label; nested
labels yield nested spans.
"""
label_at = {ln[:-1]: i for i, ln in enumerate(lines)
if ln.endswith(":")}
spans = {}
for i, ln in enumerate(lines):
if ln.endswith(":"):
continue
mnem = ln.split(None, 1)[0]
for ref in LABEL_REF.findall(ln):
if ref not in label_at:
continue
j = label_at[ref]
if j < i: # label above: backward branch
lo, hi = j, i
elif mnem.startswith("loop"): # Xtensa: end label below
lo, hi = i + 1, j - 1
else:
continue
if ref in spans: # several edges to one label
lo = min(lo, spans[ref][0])
hi = max(hi, spans[ref][1])
spans[ref] = (lo, hi)
return [(ref, lo, hi) for ref, (lo, hi)
in sorted(spans.items(), key=lambda kv: kv[1])]
def loop_spans(lines):
"""Return [(label, insns)] spans for cleaned asm lines.
The count states how many instructions lie in the span — nothing
about trip count or hotness, which the reader must judge from the
source.
"""
result = []
for ref, lo, hi in loop_span_ranges(lines):
insns = sum(1 for ln in lines[lo:hi + 1] if not ln.endswith(":"))
result.append((ref, insns))
return result
# --span-stats buckets, by mnemonic table. Covers the targets the tool
# routinely meets: Xtensa, RISC-V, and ARM by exact (or dot-stripped)
# mnemonic; x86 AT&T by the memory-operand heuristic in classify_insn,
# since there loads and stores are mostly spellings of mov.
_LOAD_MNEMONICS = frozenset("""
l8ui l16ui l16si l32i l32i.n l32r l32e l32ai lsi lsiu lsx lsxu
lb lbu lh lhu lw lwu ld flw fld c.lw c.lwsp c.ld c.ldsp c.flw c.fld
ldr ldrb ldrh ldrsb ldrsh ldrd ldm ldmia vldr vldm
pop popl popq
""".split())
_STORE_MNEMONICS = frozenset("""
s8i s16i s32i s32i.n s32e s32ri s32c1i ssi ssiu ssx ssxu
sb sh sw sd fsw fsd c.sw c.swsp c.sd c.sdsp c.fsw c.fsd
str strb strh strd stm stmia vstr vstm
push pushl pushq
""".split())
# Any multiply, including multiply-accumulate and FP forms (mull,
# mulsh, mula.dd.*, fmadd.s, vmla.f32, imull, mulss, ...).
_MUL_PREFIXES = ("mul", "imul", "fmul", "fmadd", "fmsub", "fnmadd",
"fnmsub", "madd", "msub", "mla", "mls", "smul", "umul",
"smla", "umla", "vmul", "vmla", "vmls", "vfma", "vfms")
_BRANCH_MNEMONICS = frozenset("""
beq bne blt bge bltu bgeu beqz bnez beqi bnei blti bgei bltui bgeui
bbci bbsi bbc bbs bany bnone ball bnall bt bf
blez bgez bltz bgtz bgt ble bgtu bleu
b bl bx blx cbz cbnz bcs bcc bmi bpl bvs bvc bhi bls bal
jr jal jalr ret retw tail
c.j c.jr c.jal c.jalr c.beqz c.bnez
""".split())
def classify_insn(line):
"""Bucket one cleaned instruction: load/store/mul/branch/other.
Branch means any control transfer, calls included (a span's
outbound calls are already itemised in the calls column). For
mnemonics no table knows, an AT&T-style memory operand decides
load (source side) vs store (destination, the last operand);
lea and alignment nops are excluded from that heuristic first.
"""
parts = line.split(None, 1)
mnem = parts[0].lower()
rest = parts[1] if len(parts) > 1 else ""
base = mnem.split(".", 1)[0]
if mnem in _LOAD_MNEMONICS or base in _LOAD_MNEMONICS:
return "load"
if mnem in _STORE_MNEMONICS or base in _STORE_MNEMONICS:
return "store"
if base.startswith(_MUL_PREFIXES):
return "mul"
if mnem in _BRANCH_MNEMONICS or base in _BRANCH_MNEMONICS:
return "branch"
if base.startswith(("j", "call", "loop")):
return "branch"
if base.startswith(("lea", "nop")):
return "other"
if "(" in rest:
depth, cut = 0, -1
for i, ch in enumerate(rest):
if ch == "(":
depth += 1
elif ch == ")":
depth -= 1
elif ch == "," and depth == 0:
cut = i
return "store" if "(" in rest[cut + 1:] else "load"
return "other"
_MIX_KEYS = ("load", "store", "mul", "branch", "other")
def span_mix(lines):
"""[{label, insns, load, store, mul, branch, other}] per loop span
of one function — the data behind --span-stats, table and JSON."""
result = []
for label, lo, hi in loop_span_ranges(lines):
body = [ln for ln in lines[lo:hi + 1] if not ln.endswith(":")]
counts = dict.fromkeys(_MIX_KEYS, 0)
for ln in body:
counts[classify_insn(ln)] += 1
entry = {"label": label, "insns": len(body)}
entry.update(counts)
result.append(entry)
return result
def span_stats_table(fn_names, funcs, max_width=None):
"""Per-span instruction mix for the named functions.
One row per loop span: how many of its instructions load, store,
multiply, or branch. This weighs the span the way the doctrine
asks — the hand-written awk this replaces kept counting past the
loop end into the epilogue.
"""
rows = [("function", "span", "insns") + _MIX_KEYS]
for name in fn_names:
for mix in span_mix(funcs[name]):
rows.append((name, mix["label"], str(mix["insns"]))
+ tuple(str(mix[k]) for k in _MIX_KEYS))
if len(rows) == 1:
return "(no loop spans)"
return render_table(rows, max_width)
def auto_pairs(names):
"""Pair old_X with new_X for every X present in both."""
names = list(names)
return [(n, "new_" + n[4:]) for n in names
if n.startswith("old_") and "new_" + n[4:] in names]
# Suffixes that make a nonexistent positional read as a mistyped source
# file rather than a function name to inspect.
SOURCE_SUFFIXES = {".c", ".h", ".i", ".s", ".cc", ".cpp", ".cxx", ".hpp"}
def split_positionals(positionals):
"""Split positional arguments into source files and function names.
The first positional is always a source. A later one that exists
on disk is a source too; a bare name is a function to inspect; a
path-looking argument that does not exist (a separator, or a source
suffix) is a mistyped file - exiting beats searching the assembly
for a symbol named like a filename.
"""
sources, fn_names = positionals[:1], []
for arg in positionals[1:]:
if Path(arg).exists():
sources.append(arg)
elif ("/" in arg or os.sep in arg
or Path(arg).suffix.lower() in SOURCE_SUFFIXES):
sys.exit(f"error: no such file: {arg} (a function name to "
"inspect must be a bare symbol, not a path)")
else:
fn_names.append(arg)
return sources, fn_names
def is_elf(path):
"""True when path is an ELF binary - by magic bytes, not extension,
so a stripped-suffix or oddly named firmware image still routes to
ELF mode and a text file named *.elf does not."""
try:
with open(path, "rb") as fh:
return fh.read(4) == b"\x7fELF"
except OSError:
return False
def find_config(explicit, sources):
"""Locate the config file; first hit wins, no merging.
Order: --config PATH, then asmdiff.toml next to the first source
file (a harness directory can carry its own targets), then the
current directory, then ~/.config/asmdiff.toml.
"""
if explicit:
path = Path(explicit)
if not path.is_file():
sys.exit(f"error: config file not found: {explicit}")
return path
for candidate in (Path(sources[0]).resolve().parent / CONFIG_NAME,
Path.cwd() / CONFIG_NAME,
Path.home() / ".config" / CONFIG_NAME):
if candidate.is_file():
return candidate
return None
def load_config(path):
"""Parse a TOML config: one [table] per target, optional top-level
`default` naming the target(s) to run when no --cc/--target is given."""
if tomllib is None:
sys.exit(f"error: {path} exists but this Python has no tomllib "
"(config files need Python >= 3.11)")
try:
with open(path, "rb") as fh:
return tomllib.load(fh)
except tomllib.TOMLDecodeError as exc:
sys.exit(f"error: {path}: {exc}")
def resolve_editor(env=None):
"""$VISUAL, then $EDITOR, split so values like 'code -w' work; on
Windows fall back to notepad (neither variable is normally set
there), elsewhere an unset editor is an error, never a guess."""
env = os.environ if env is None else env
editor = env.get("VISUAL") or env.get("EDITOR")
if editor:
return shlex.split(editor)
if os.name == "nt":
return ["notepad"]
sys.exit("error: set $VISUAL or $EDITOR to edit the config")
def edit_config(explicit):
"""Open the config in the user's editor, creating it first from
EXAMPLE_CONFIG when missing (a pip/uvx install has no example file
on disk). With --config PATH that file is edited; otherwise the
global fallback location find_config searches last. The TOML check
afterwards is warn-only: a half-finished edit should not eat the
editor's exit status."""
path = (Path(explicit) if explicit
else Path.home() / ".config" / CONFIG_NAME)
if not path.is_file():