Implement AD testing and benchmarking (hand rolled)

[penelopeysm]

One of two options. The other one to come.

This PR implements functionality for testing and benchmarking AD. It is largely copied over from my ModelTests repo where I've been playing around with this.

Closes #869

What does it contain?

It basically adds one function DynamicPPL.TestUtils.AD.run_ad. See the docstring for more info.

Why not an extension?

The only new dependencies are Statistics, which is stdlib, and Chairmarks, which itself has no non-stdlib dependencies. I therefore consider it unnecessary to add an extension (which would bring a number of drawbacks, e.g. reduced discoverability as users have to load the trigger packages themselves, us having to faff around with functions declared in src/ and extended in ext/, ...)

[github-actions]

Benchmark Report for Commit b317ab2

Computer Information

Julia Version 1.11.4
Commit 8561cc3d68d (2025-03-10 11:36 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Benchmark Results

|                 Model | Dimension |  AD Backend |      VarInfo Type | Linked | Eval Time / Ref Time | AD Time / Eval Time |
|-----------------------|-----------|-------------|-------------------|--------|----------------------|---------------------|
| Simple assume observe |         1 | forwarddiff |             typed |  false |                  9.5 |                 1.7 |
|           Smorgasbord |       201 | forwarddiff |             typed |  false |                607.8 |                41.5 |
|           Smorgasbord |       201 | forwarddiff | simple_namedtuple |   true |                424.7 |                45.5 |
|           Smorgasbord |       201 | forwarddiff |           untyped |   true |               1230.1 |                27.8 |
|           Smorgasbord |       201 | forwarddiff |       simple_dict |   true |               3727.8 |                20.9 |
|           Smorgasbord |       201 | reversediff |             typed |   true |               1453.4 |                28.9 |
|           Smorgasbord |       201 |    mooncake |             typed |   true |                952.5 |                 5.3 |
|    Loop univariate 1k |      1000 |    mooncake |             typed |   true |               5596.5 |                 4.0 |
|       Multivariate 1k |      1000 |    mooncake |             typed |   true |               1080.1 |                 8.5 |
|   Loop univariate 10k |     10000 |    mooncake |             typed |   true |              62004.3 |                 3.7 |
|      Multivariate 10k |     10000 |    mooncake |             typed |   true |               8940.4 |                 9.7 |
|               Dynamic |        10 |    mooncake |             typed |   true |                139.2 |                11.9 |
|              Submodel |         1 |    mooncake |             typed |   true |                 26.1 |                 9.3 |
|                   LDA |        12 | reversediff |             typed |   true |                525.8 |                 5.2 |

[codecov]

Codecov Report

Attention: Patch coverage is 68.96552% with 9 lines in your changes missing coverage. Please review.

Project coverage is 84.75%. Comparing base (eed80e5) to head (b317ab2).

Files with missing lines	Patch %	Lines
src/test_utils/ad.jl	68.96%	9 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #882      +/-   ##
==========================================
- Coverage   84.87%   84.75%   -0.13%     
==========================================
  Files          34       35       +1     
  Lines        3815     3844      +29     
==========================================
+ Hits         3238     3258      +20     
- Misses        577      586       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coveralls]

Pull Request Test Coverage Report for Build 14256002347

Details

• 20 of 29 (68.97%) changed or added relevant lines in 1 file are covered.
• 1213 unchanged lines in 26 files lost coverage.
• Overall coverage decreased (-0.1%) to 84.844%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/test_utils/ad.jl	20	29	68.97%

Files with Coverage Reduction	New Missed Lines	%
src/DynamicPPL.jl	1	90.91%
src/submodel_macro.jl	2	82.35%
src/extract_priors.jl	5	53.57%
src/test_utils/model_interface.jl	5	22.22%
src/test_utils/varinfo.jl	5	78.26%
src/values_as_in_model.jl	6	43.9%
src/logdensityfunction.jl	7	64.44%
src/model_utils.jl	7	7.14%
src/distribution_wrappers.jl	13	9.76%
src/model.jl	13	80.0%

Totals
Change from base Build 14127923718:	-0.1%
Covered Lines:	3258
Relevant Lines:	3840

---

💛 - Coveralls