attempt to fix warmup bookkeeping: dropped the tune stat #8015
Conversation
|
@OriolAbril / @aloctavodia does any part of Arviz require the step samples to have a tune flag? Is it enough that we have warmup / posterior distinction, each with their number of draws? |
| return steps | ||
|
|
||
|
|
||
| def check_step_emits_tune(step: CompoundStep | BlockedStep): |
There was a problem hiding this comment.
If this ends up working without tune, remove this function as well
| ) -> tuple[mcb.RunMeta, PointFunc]: | ||
| variables, point_fn = get_variables_and_point_fn(model, initial_point) | ||
|
|
||
| check_step_emits_tune(step) |
There was a problem hiding this comment.
@michaelosthege what's the requirement here? Can we change things on mcbackend to not expect tune info attached to the steps?
There was a problem hiding this comment.
No we can not change that tune info must be attached to a step. See my comment in #7997 (comment).
| @staticmethod | ||
| def _progressbar_config(n_chains=1): | ||
| columns = [ | ||
| TextColumn("{task.fields[tune]}", table_column=Column("Tuning", ratio=1)), |
There was a problem hiding this comment.
We'll need a work-around for this, as we still want to show whether we're in tuning or not in the progressbar
|
Taking a step back, would it make sense for a Even if that's the case, I think it still makes sense to remove this currently useless stat and reintroduce in a separate PR (provided nobody finds a reason why it is actually useful/needed). |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8015 +/- ##
==========================================
+ Coverage 90.22% 90.85% +0.63%
==========================================
Files 116 116
Lines 18972 18961 -11
==========================================
+ Hits 17117 17227 +110
+ Misses 1855 1734 -121
🚀 New features to boost your workflow:
|
Automatically stopping the warmup early would be nice. I think we should agree on cleanly separated definitions of warmup, burn-in and tuning. Samplers not needing to tune parameters doesn't mean that there's no need for a warmup phase of burn-in iterations (however one might call it). Our current implementation is bad because it doesn't separate the concepts. |
3 participants
Description
I tried to make “tuning vs draws” a driver owned concept again. Right now, parts of sampling/postprocessing infer warmup length from a per-step
"tune"sampler stat, which can get out of sync (e.g. a step method returning"tune": Falseeverywhere makes PyMC thinkn_tune == 0, so warmup isn’t discarded and the logs look wrong).Related Issues
Fixes: #7997
Context: #7776 (progressbar/stat refactor that exposed the mismatch)
Related discussion/attempts: #7730, #7721, #7724, #8014