Skip to content

Release v0.37 #901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 26 commits into
base: main
Choose a base branch
from
Draft

Release v0.37 #901

wants to merge 26 commits into from

Conversation

mhauru
Copy link
Member

@mhauru mhauru commented Apr 24, 2025

0.37 checklist

Accumulators

Accumulators broke Enzyme. This is a known issue and already reported upstream:

Remove context from evaluate!! and simplify submodel code

Replacing SamplingContext with InitContext

Fixes to AD testing

Shuffle code around

  • Break up contexts and possibly accumulator code

Possibly for this version or next minor version?


Not for this version (?)

Things that can be patch releases

@mhauru mhauru marked this pull request as draft April 24, 2025 16:47
Copy link

codecov bot commented Apr 24, 2025

Codecov Report

Attention: Patch coverage is 83.43949% with 130 lines in your changes missing coverage. Please review.

Project coverage is 80.35%. Comparing base (ce7c8b1) to head (e60eab0).

Files with missing lines Patch % Lines
src/simple_varinfo.jl 62.00% 19 Missing ⚠️
src/test_utils/ad.jl 0.00% 16 Missing ⚠️
src/varinfo.jl 88.96% 16 Missing ⚠️
src/abstract_varinfo.jl 81.33% 14 Missing ⚠️
src/default_accumulators.jl 84.14% 13 Missing ⚠️
src/threadsafe.jl 73.91% 12 Missing ⚠️
src/debug_utils.jl 90.27% 7 Missing ⚠️
src/extract_priors.jl 75.00% 6 Missing ⚠️
src/logdensityfunction.jl 68.42% 6 Missing ⚠️
ext/DynamicPPLJETExt.jl 0.00% 4 Missing ⚠️
... and 7 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #901      +/-   ##
==========================================
- Coverage   82.97%   80.35%   -2.62%     
==========================================
  Files          36       38       +2     
  Lines        3965     4017      +52     
==========================================
- Hits         3290     3228      -62     
- Misses        675      789     +114     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coveralls
Copy link

coveralls commented Apr 24, 2025

Pull Request Test Coverage Report for Build 16379148064

Details

  • 655 of 785 (83.44%) changed or added relevant lines in 27 files are covered.
  • 61 unchanged lines in 11 files lost coverage.
  • Overall coverage decreased (-0.8%) to 82.359%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/utils.jl 7 8 87.5%
src/context_implementations.jl 31 33 93.94%
src/experimental.jl 0 2 0.0%
src/pointwise_logdensities.jl 49 51 96.08%
src/accumulators.jl 29 32 90.63%
src/model.jl 31 34 91.18%
ext/DynamicPPLJETExt.jl 0 4 0.0%
src/values_as_in_model.jl 20 24 83.33%
src/extract_priors.jl 18 24 75.0%
src/logdensityfunction.jl 13 19 68.42%
Files with Coverage Reduction New Missed Lines %
src/context_implementations.jl 1 93.33%
src/debug_utils.jl 1 88.56%
src/test_utils/contexts.jl 1 88.89%
src/contexts.jl 2 74.45%
src/utils.jl 2 74.06%
src/logdensityfunction.jl 3 61.54%
src/distribution_wrappers.jl 6 28.57%
src/simple_varinfo.jl 7 71.26%
src/abstract_varinfo.jl 9 76.47%
src/threadsafe.jl 11 60.0%
Totals Coverage Status
Change from base Build 16174747412: -0.8%
Covered Lines: 3310
Relevant Lines: 4019

💛 - Coveralls

mhauru and others added 5 commits May 2, 2025 14:12
* Release 0.36

* AbstractPPL 0.11 + change prefixing behaviour (#830)

* AbstractPPL 0.11; change prefixing behaviour

* Use DynamicPPL.prefix rather than overloading

* Remove VarInfo(VarInfo, params) (#870)

* Unify `{untyped,typed}_{vector_,}varinfo` constructor functions (#879)

* Unify {Untyped,Typed}{Vector,}VarInfo constructors

* Update invocations

* NTVarInfo

* Fix tests

* More fixes

* Fixes

* Fixes

* Fixes

* Use lowercase functions, don't deprecate VarInfo

* Rewrite VarInfo docstring

* Fix methods

* Fix methods (really)

* Draft of accumulators

* Fix some variable names

* Fix pointwise_logdensities, gut tilde_observe, remove resetlogp!!

* Map rather than broadcast

Co-authored-by: Tor Erlend Fjelde <[email protected]>

* Start documenting accumulators

* Use Val{symbols} instead of AccTypes to index

* More documentation for accumulators

* Link varinfo by default in AD testing utilities; make test suite run on linked varinfos (#890)

* Link VarInfo by default

* Tweak interface

* Fix tests

* Fix interface so that callers can inspect results

* Document

* Fix tests

* Fix changelog

* Test linked varinfos

Closes #891

* Fix docstring + use AbstractFloat

* Fix resetlogp!! and type stability for accumulators

* Fix type rigidity of LogProbs and NumProduce

* Fix uses of getlogp and other assorted issues

* setaccs!! nicer interface and logdensity function fixes

* Revert back to calling the macro @addlogprob!

* Remove a dead test

* Clarify a comment

* Implement split/combine for PointwiseLogdensityAccumulator

* Switch ThreadSafeVarInfo.accs_by_thread to be a tuple

* Fix `condition` and `fix` in submodels (#892)

* Fix conditioning in submodels

* Simplify contextual_isassumption

* Add documentation

* Fix some tests

* Add tests; fix a bunch of nested submodel issues

* Fix fix as well

* Fix doctests

* Add unit tests for new functions

* Add changelog entry

* Update changelog

Co-authored-by: Hong Ge <[email protected]>

* Finish docs

* Add a test for conditioning submodel via arguments

* Clean new tests up a bit

* Fix for VarNames with non-identity lenses

* Apply suggestions from code review

Co-authored-by: Markus Hauru <[email protected]>

* Apply suggestions from code review

* Make PrefixContext contain a varname rather than symbol (#896)

---------

Co-authored-by: Hong Ge <[email protected]>
Co-authored-by: Markus Hauru <[email protected]>

* Revert ThreadSafeVarInfo back to Vectors and fix some AD type casting in (Simple)VarInfo

* Improve accumulator docs

* Add test/accumulators.jl

* Docs fixes

* Various small fixes

* Make DynamicTransformation not use accumulators other than LogPrior

* Fix variable order and name of map_accumulator!!

* Typo fixing

* Small improvement to ThreadSafeVarInfo

* Fix demo_dot_assume_observe_submodel prefixing

* Typo fixing

* Miscellaneous small fixes

* HISTORY entry and more miscellanea

* Add more tests for accumulators

* Improve accumulators docstrings

* Fix a typo

* Expand HISTORY entry

* Add accumulators to API docs

* Remove unexported functions from API docs

* Add NamedTuple methods for get/set/acclogp

* Fix setlogp!! with single scalar to error

* Export AbstractAccumulator, fix a docs typo

* Apply suggestions from code review

Co-authored-by: Penelope Yong <[email protected]>

* Rename LogPrior -> LogPriorAccumulator, and Likelihood and NumProduce

* Type bound log prob accumulators with T<:Real

* Add @addlogprior! and @addloglikelihood!

* Apply suggestions from code review

Co-authored-by: Penelope Yong <[email protected]>

* Move default accumulators to default_accumulators.jl

* Fix some tests

* Introduce default_accumulators()

* Go back to only having @addlogprob!

* Fix tilde_observe!! prefixing

* Fix default_accumulators internal type

* Make unflatten more type stable, and add a test for it

* Always print all benchmark results

* Move NumProduce VI functions to abstract_varinfo.jl

---------

Co-authored-by: Penelope Yong <[email protected]>
Co-authored-by: Tor Erlend Fjelde <[email protected]>
Co-authored-by: Hong Ge <[email protected]>
* Implement values_as_in_model using an accumulator

* Make make_varname_expression a function

* Refuse to combine ValuesAsInModelAccumulators with different include_colon_eqs

* Fix nested context test
Copy link
Contributor

github-actions bot commented Jun 2, 2025

Benchmark Report for Commit e60eab0

Computer Information

Julia Version 1.11.6
Commit 9615af0f269 (2025-07-09 12:58 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Benchmark Results

|                 Model | Dimension |  AD Backend |      VarInfo Type | Linked | Eval Time / Ref Time | AD Time / Eval Time |
|-----------------------|-----------|-------------|-------------------|--------|----------------------|---------------------|
| Simple assume observe |         1 | forwarddiff |             typed |  false |                 13.0 |                 1.4 |
|           Smorgasbord |       201 | forwarddiff |             typed |  false |                734.5 |                36.8 |
|           Smorgasbord |       201 | forwarddiff | simple_namedtuple |   true |                469.4 |                47.8 |
|           Smorgasbord |       201 | forwarddiff |           untyped |   true |               1041.4 |                33.5 |
|           Smorgasbord |       201 | forwarddiff |       simple_dict |   true |               6445.5 |                23.6 |
|           Smorgasbord |       201 | reversediff |             typed |   true |               1521.5 |                26.7 |
|           Smorgasbord |       201 |    mooncake |             typed |   true |               1086.8 |                12.2 |
|    Loop univariate 1k |      1000 |    mooncake |             typed |   true |               6763.0 |                42.5 |
|       Multivariate 1k |      1000 |    mooncake |             typed |   true |                998.4 |                 8.7 |
|   Loop univariate 10k |     10000 |    mooncake |             typed |   true |              81414.0 |                39.9 |
|      Multivariate 10k |     10000 |    mooncake |             typed |   true |               8555.3 |                 9.7 |
|               Dynamic |        10 |    mooncake |             typed |   true |                138.0 |                30.1 |
|              Submodel |         1 |    mooncake |             typed |   true |                 16.9 |                22.5 |
|                   LDA |        12 | reversediff |             typed |   true |               1126.2 |                 1.9 |

Copy link
Contributor

github-actions bot commented Jun 2, 2025

DynamicPPL.jl documentation for PR #901 is available at:
https://TuringLang.github.io/DynamicPPL.jl/previews/PR901/

@mhauru mhauru mentioned this pull request Jun 3, 2025
…952)

* Change `evaluate!!` API, add `sample!!`

* Fix literally everything else that I broke

* Fix some docstrings

* fix ForwardDiffExt (look, multiple dispatch bad...)

* Changelog

* fix a test

* Fix docstrings

* use `sample!!`

* Fix a couple more cases

* Globally rename `sample!!` -> `evaluate_and_sample!!`, add changelog warning
* Move submodel code to submodel.jl

* Remove `@submodel`
* Clean up submodel code, remove 3-arg `_evaluate!!`

* Remove 3-argument `evaluate!!` as well

* Update changelog

* Improve submodel error message

* Fix doctest

* Add error hint for three-argument evaluate!!
* Rework API for AD testing

* Fix test

* Add `rng` keyword argument

* Use atol and rtol

* remove unbound type parameter (?)

* Don't need to do elementwise check

* Update changelog

* Fix typo
penelopeysm and others added 4 commits July 9, 2025 18:38
* DebugContext -> DebugAccumulator

* Changelog

* Force `conditioned` to return a dict

* fix conditioned implementation

* revert `conditioned` bugfix (will merge this to main instead)

* fix show

* Fix doctests

* fix doctests 2

* Make VarInfo actually mandatory in check_model

* Re-implement `missing` check

* Revert `combine` signature in docstring

* Revert changes to `Base.show` on AccumulatorTuple

* Add TODO comment about VariableOrderAccumulator

Co-authored-by: Markus Hauru <[email protected]>

* Fix doctests

---------

Co-authored-by: Markus Hauru <[email protected]>
* Turn NumProduceAccumulator into VariableOrderAccumulator

* Add comparison methods

* Make VariableOrderAccumulator use regular Dict

* Use copy rather than deepcopy for accumulators

* Minor docstring touchup

* Remove unnecessary use of NumProduceAccumulator

* Fix split(VariableOrderAccumulator)

* Remove NumProduceAcc from Debug

* Fix set_retained_vns_del!

---------

Co-authored-by: Penelope Yong <[email protected]>
* Give LogDensityFunction the getlogdensity field

* Allow missing LogPriorAccumulator when linking

* Trim whitespace

* Run formatter

* Fix a few typos

* Fix comma -> semicolon

* Fix `LogDensityAt` invocation

* Fix one last test

* Fix tests

---------

Co-authored-by: Penelope Yong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants