Add Func::approximate_by() and composable Approximations by alexreinking · Pull Request #9486 · halide/Halide

alexreinking · 2026-09-29T21:36:05Z

Adds Func::approximate_by() and composable Approximations, a way to splice a lossy transform such as a quantize/dequantize round trip into an existing call graph. Like rfactor() or a targeted Func::in(), approximate_by() eagerly replaces calls to a Func within the given consumers, here with decode(encode(f)).

Units are plain structs with encode()/decode(), held by a type-erased Approximation; Pointwise builds an elementwise unit from two lambdas.
Combinators: Compose, Parallel (positional or by port name), Permute, Choose, Identity, and the TrustedInverse escape hatch.
Layout units: BlockReshape (including multi-dimensional BlockReshape::tiles), StructLayout, PlanarFieldPack, LittleEndianScalarPack, all rank-polymorphic over trailing batch dimensions.
Ports and signatures: named wires; declared types, dimensions and ranges are checked at every call; describe() and check_ranges().
Results: a trace tree, encoded_by/decoded_by lookups, the encoded boundary for Pipeline::sever(), and decode_funcs() for eager_inline().
Tools (header-only): make_codec() builds encode/decode generators from one Approximation (using the new Generator::add_output(name, Func)); halide_approximation_testing.h provides seeded inputs, round-trip error reports and a property library.

The design rationale is in doc/Approximation.md. Tutorial lesson 25 builds GGML's Q4_0 from these pieces and checks it bit-for-bit against GGML's quantizer. This is a research-mode API.

Part of stack #9540. Uses Stage::distribute() (#9508) and the eager_inline() contract (#9510); the GGML app (#9511) is the motivating user.

Tests: correctness_approximate_by, correctness_approximation_*, generator_aot_approximation_codec, tutorial lesson 25.

Authored by GitHub Copilot

codecov · 2026-09-29T23:56:15Z

Codecov Report

❌ Patch coverage is 80.66492% with 221 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.35%. Comparing base (79a1fa7) to head (0baf64e).

Files with missing lines	Patch %	Lines
src/Approximation.cpp	82.34%	101 Missing and 84 partials ⚠️
src/Approximation.h	41.17%	19 Missing and 1 partial ⚠️
src/Func.cpp	73.77%	11 Missing and 5 partials ⚠️

Additional details and impacted files

@@                             Coverage Diff                              @@
##           alexreinking/multiramp-alignment-stacked    #9486      +/-   ##
============================================================================
+ Coverage                                     71.03%   71.35%   +0.31%     
============================================================================
  Files                                           262      264       +2     
  Lines                                         81609    82730    +1121     
  Branches                                      19928    20215     +287     
============================================================================
+ Hits                                          57974    59032    +1058     
- Misses                                        17614    17670      +56     
- Partials                                       6021     6028       +7

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

approximate_by() eagerly and destructively replaces every call to a Func inside a set of consumers with a call to the round trip decode(encode(f)) -- a lossy, quantified Func-to-Func transform where decode(encode(f)) reproduces f's signature. The substitution happens immediately, like rfactor() and the targeted form of Func::in(), but substitutes a different computation rather than an identity wrapper. The transform is expressed as an Approximation: a bidirectional encode()/decode() pair (src/Approximation.{h,cpp}), with combinators -- Compose, Apply, Permute, Choose, Identity, and the TrustedInverse escape hatch -- for building up a codec from smaller pieces. Each Approximation carries a stage key, and approximate_by() records every stage's encoded and decoded outputs, so callers can find and schedule a specific stage's Funcs after composition. src/ApproximationComponents.h provides reusable building blocks: block reshaping, struct layouts, storage casts, bit and field packing, and symmetric block quantization. See doc/ApproximationDesign.md for the design rationale. Verified by test/correctness/approximate_by.cpp and test/correctness/approximation_components.cpp. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Approximation is now a type-erased handle (in the style of std::function) rather than a virtual base class. Any type with const encode()/decode() methods converts to it implicitly, so leaf units and combinators are plain structs. Combinators hold children as Approximation values instead of unique_ptr, and ComposeBuilder, approximation_ptr, and ApproximationStageKey are gone. Copies of a handle are the same stage; converting a unit twice makes two distinct stages. Every stage invoked through a handle is recorded in the result's stage outputs automatically, so combinators and approximate_by() no longer push records by hand, and ApproximationResult::encoded_by() / decoded_by() now take the handle itself. Looking up a stage that was not invoked in that direction, or was invoked more than once, is an error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Per direction, a unit may now provide just `Func encode(const Func &)` or `std::vector<Func> encode(const std::vector<Func> &)` instead of the full EncodeResult form (and likewise for decode); the handle fills in empty handles and stage outputs. Pointwise builds an elementwise unit from a pair of Expr (or Tuple) functions. StorageCast and AdditiveOffset are now expressed with Pointwise, keeping their Func names. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Units now return only Funcs (single or vector form); the handle finds each stage's intermediates itself -- every Func reachable from the stage's outputs without passing through its inputs, in producer-first order -- and records them per stage. EncodeResult/DecodeResult/ ApproximationResult rename `handles` to `intermediates`, since "handle" now means an Approximation. Stage traces propagate through a collector that the outermost handle call opens, so combinators (and any user unit that calls other handles) are ordinary units and no longer forward child traces by hand. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Handles carry a label (explicit via labelled(), else the unit's name(), else its demangled type name), shared by every copy. Each encode/decode records a tree of the handle calls made beneath it, available as `trace` on EncodeResult/DecodeResult and encode_trace/decode_trace on ApproximationResult, and printable via operator<<. The flat stage-output lists are now derived from the tree. ApproximationResult::stage_ports() returns every stage-boundary Func for scheduling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Units may declare an ApproximationSignature (statically, or contextually from their input ports for combinators) naming their ports and optionally constraining each port's type and dimensionality; the handle validates Funcs against it in both directions. Port names flow through Compose, Apply, Identity, Permute, and Choose, and are recorded in the trace, so Apply can select a port by name (with arities taken from the inner stage) and results can be queried with encoded_by(stage, "name"). Undeclared units pass names through when arity is preserved and fall back to positional names otherwise. describe() (and operator<<) renders an Approximation's structure and signatures without running it. The core components declare signatures where their port types are known. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ions ApproximationTesting.h (header-only, in Halide.h) provides seeded, platform-independent input distributions (including adversarial ones), verify_round_trip() error reports, and a property library (lossless, bounded_error, within_declared_bound, idempotent_requantize, zero- and sign-preservation, outputs_within_declared_ranges) run by check_property() over seeded trials, with the failing seed reported for reproduction. Properties can be conditioned on input ranges: ports may declare value ranges (preconditions on encode inputs, guarantees on encode outputs), default generators respect them, check_ranges()/describe() flag unmet preconditions statically, and Property::at(stage) checks one stage of a composition on the values that actually reach it from upstream. Units may also declare lossless() or a per-element error_bound(); the core components declare these where they hold. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Renames doc/ApproximationDesign.md to doc/Approximation.md, rewrites it to describe the API as it now exists (the duck-typed handle, unit forms, discovered intermediates, the trace tree, ports and signatures, and ApproximationTesting.h), and links it from the Guides toctree. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Decoding took its output port names from the unit's signature resolved without an encode-side context, so Permute (whose context-free signature is positional) lost the names it received, and a by-name Apply after it failed in decode. Units may now provide decoded_ports(encoded); Permute, Identity, Apply, Compose, TrustedInverse and Choose do. The testing helpers could not read back Tuple-valued or struct-typed encoded Funcs but still dereferenced them. Properties needing encoded values now fail with a clear message instead; the others still apply. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

It is a test-time helper that no pipeline needs, so it now lives next to halide_image_io.h as tools/halide_approximation_testing.h, installed with Halide::Tools, rather than in the monolithic Halide.h. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Apply's positional and by-name forms are replaced by Parallel, a product combinator: Parallel{a, b} gives each child a consecutive slice sized by its signature, and Parallel{{"codes", a}, {"scale", b}} routes ports by name, passing the rest through in place. Port names now identify a wire in both directions: names flowing from upstream win over a unit's declared defaults, and decode's outputs mirror encode's inputs, including for a stand-alone decode. This makes by-name routing work across StructLayout and removes the decoded_ports() hook. Compose now lists stages in encode order (innermost first). Also: the testing header compiles against a plain Halide.h, and SymmetricBlockQuantize's TruncateHalfUpWithOffset rounding adds qmax + 0.5 in one step, as GGML does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Builds GGML's Q4_0 block format from Approximation components, splices it into a dot product with approximate_by, splits the quantizer off with compute_offline, and checks it bit-for-bit against a C++ transcription of GGML's reference quantizer, plus property checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Quantization math (SymmetricBlockQuantize, AdditiveRadixSplit, BinaryAlphabetPack) is GGML-specific and belongs in client code. StorageCast and AdditiveOffset are dropped in favor of inline Pointwise units, which gain with_types(), with_ranges() and with_lossless(). The remaining generic layout units (BlockReshape, StructLayout, PlanarFieldPack, LittleEndianScalarPack) move into Approximation.h/.cpp, and ApproximationComponents.h is deleted. Tests and the tutorial define their own quantizers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Approximation default labels used typeid(T) unconditionally, which breaks consumers built with -fno-rtti. Fall back to a generic label. parallel_fork.cpp declared an unscoped enum value Parallel under using namespace Halide, which is now ambiguous with Halide::Parallel. Make the enum scoped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Derive unit default labels from __PRETTY_FUNCTION__/__FUNCSIG__ instead of typeid so -fno-rtti builds keep real type names. Skip error-path tests when exceptions are disabled, use relative float tolerances in approximate_by, and address clang-tidy findings in Approximation and lesson 25. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

BlockReshape, PlanarFieldPack and StructLayout now pass trailing dimensions through unchanged, so one scheme value approximates a row or a matrix of rows. Their signatures are contextual: dimensionalities follow the input ports, and are unknown without context. StructLayout's record_dimensions is now optional and inferred from its inputs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

decode_funcs() lists the decode side's Funcs that eager_inline() accepts as they are scheduled now (replacement and the decode root's intermediates, producers first), so one consumer.eager_inline() call folds the inlinable decode chain in. The eager_inline() checks are factored into Internal::eager_inline_obstacle(), which returns the reason rather than erroring; eager_inline()'s messages are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

make_codec() packages the configure()/sever()/adopt pattern for codec Generators: it approximates an ImageParam of values by a scheme, gives the result a default schedule, severs it at the encoded Funcs, and returns both halves with stable port names. adopt_encoder() and adopt_decoder() declare either half's ports on a Generator. Header-only, so it adds no library API. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Leave stage ports inline by default, as Halide does; the encode side's stage boundaries need not be materialized for a codec. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…d dims Func names are made unique per process, so a codec Generator's configure() renamed its Funcs (decoded$1, codes$2) on its second run in a multi-target build, and the codec helper rejected them. Add GeneratorBase::add_output(const std::string &name, const Func &existing), have Codec record its port names and adopt with them, and build the approximation_codec AOT test multi-target to cover it. StructLayout now rejects an inferred record dimensionality below one (e.g. a 0-D scalar slot) with a user error instead of crashing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

BlockReshape::tiles({b0, b1, ...}) splits each leading dimension by its own block, within-tile indices first: (x0, x1, rest...) <-> (x0 % b0, x1 % b1, x0 / b0, x1 / b1, rest...). Trailing dimensions pass through; types and ranges pass through; lossless. tiles({b}) is BlockReshape(b). Lets a schedule vectorize across a whole tile, e.g. the 2x2 sub-tiles of a matrix-multiply register block. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The 2x2 tile is one dense 4-wide store only if the output's tile rows are adjacent; without a stride constraint it lowers to two 2-wide stores. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

What's left of the within-block index for the last extent is already in range, and the % made bounds inference widen a piece of a block to the whole block (e.g. a stage at a loop over 4-element pieces of a 32-element block realized all 32). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

alexreinking added this pull request to stack #9487 September 29, 2026 21:36

alexreinking mentioned this pull request Sep 29, 2026

Add apps/ggml: Halide reimplementation of GGML quant kernels #9417

Closed

3 of 5 tasks

alexreinking force-pushed the alexreinking/approximations branch from 15927f1 to bf8c475 Compare September 29, 2026 21:40

alexreinking changed the title ~~Add Func::approximate_by() directive and Approximation combinators~~ Add Func::approximate_by() and composable Approximations Sep 30, 2026

alexreinking force-pushed the alexreinking/approximations branch from c2ea4a9 to b1a6b31 Compare September 30, 2026 05:36

alexreinking mentioned this pull request Sep 30, 2026

Filter std:: instantiations out of Halide.dll's export list #9489

Merged

2 of 5 tasks

alexreinking force-pushed the alexreinking/approximations branch 2 times, most recently from cccab1a to 891b76b Compare September 30, 2026 19:41

alexreinking force-pushed the alexreinking/approximations branch 2 times, most recently from 84a4a8d to ef2ee20 Compare October 2, 2026 17:22

alexreinking force-pushed the alexreinking/approximations branch 2 times, most recently from 03c20e6 to e319cdb Compare October 2, 2026 19:40

Base automatically changed from alexreinking/compute-offline to main October 5, 2026 17:19

alexreinking force-pushed the alexreinking/approximations branch 2 times, most recently from 9ea2609 to cdab38c Compare October 6, 2026 17:05

alexreinking removed this pull request from stack #9487 October 6, 2026 17:11

alexreinking changed the base branch from main to alexreinking/stage-distribute October 6, 2026 17:13

alexreinking added this pull request to stack #9509 October 6, 2026 17:13

alexreinking force-pushed the alexreinking/approximations branch from cdab38c to 48ea8fd Compare October 6, 2026 17:47

alexreinking removed this pull request from stack #9509 October 6, 2026 17:47

alexreinking changed the base branch from alexreinking/stage-distribute to alexreinking/eager-inline-schedules October 6, 2026 17:47

alexreinking added this pull request to stack #9512 October 6, 2026 17:47

alexreinking mentioned this pull request Oct 6, 2026

apps/ggml: GGML quantized kernels on Approximations #9511

Draft

alexreinking removed this pull request from stack #9512 October 6, 2026 19:33

alexreinking force-pushed the alexreinking/approximations branch from 48ea8fd to ca134f5 Compare October 6, 2026 19:33

alexreinking changed the base branch from alexreinking/eager-inline-schedules to alexreinking/hoist-invariants-bounds October 6, 2026 19:34

alexreinking added this pull request to stack #9522 October 6, 2026 19:34

alexreinking changed the base branch from alexreinking/simplify-update-specializations to alexreinking/multiramp-alignment-stacked October 9, 2026 07:12

alexreinking added this pull request to stack #9540 October 9, 2026 07:12

alexreinking and others added 24 commits October 9, 2026 01:47

Add Approximation to the Makefile build

404844f

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rename compute_offline to sever in approximations

436292a

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Remove version qualifiers from approximation design

5fef698

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

make_codec: compute_root only Funcs with update definitions

731d0f2

Leave stage ports inline by default, as Halide does; the encode side's stage boundaries need not be materialized for a codec. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

alexreinking force-pushed the alexreinking/approximations branch from bcd871e to df1eef6 Compare October 9, 2026 09:57

alexreinking and others added 2 commits October 10, 2026 06:20

approximation_components: constrain the tiles test's output stride

de260ea

The 2x2 tile is one dense 4-wide store only if the output's tile rows are adjacent; without a stride constraint it lowers to two 2-wide stores. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Navigation Menu

Repository navigation

Add Func::approximate_by() and composable Approximations - #9486

Add Func::approximate_by() and composable Approximations#9486
alexreinking wants to merge 26 commits into
alexreinking/multiramp-alignment-stackedfrom
alexreinking/approximations

alexreinking commented Sep 29, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Sep 29, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

alexreinking commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

alexreinking commented Sep 29, 2026 •

edited

Loading

codecov Bot commented Sep 29, 2026 •

edited

Loading