Skip to content

Resync the resampler on system time changes (take 2) #999

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 15 commits into
base: v1.x.x
Choose a base branch
from

Conversation

llucax
Copy link
Contributor

@llucax llucax commented Jul 8, 2024

This PR builds on top of #802 by using a new timer that is attached to the wall clock instead of the monotonic clock, and can be used to trigger resampling windows. It is a replacement for the channels Timer class.

This should fix the issue with the resampling windows being too small when the system clock is not in sync with the wall clock.

Status: Some code adjustments might still be needed, but for sure the big chunk of work pending are the tests, as testing this it is very complex as one need to emulate both the wall clock and the monotonic clock and the different ways they can be out of sync.

matthias-wende-frequenz and others added 15 commits July 8, 2024 10:19
Signed-off-by: Matthias Wende <[email protected]>
Adds a boolean parameter if an extra period should be added
to the calculated window end or not.
Signed-off-by: Matthias Wende <[email protected]>
Signed-off-by: Matthias Wende <[email protected]>
Co-authored-by: Leandro Lucarella <[email protected]>
Signed-off-by: Matthias Wende <[email protected]>
Co-authored-by: Leandro Lucarella <[email protected]>
Tests should try to be as close as possible as real code.

Signed-off-by: Leandro Lucarella <[email protected]>
Future changes require adding more code to this module, which is already
quite big. This commit makes it a package and split the module into a
few sub-modules to make it more manageable.

Signed-off-by: Leandro Lucarella <[email protected]>
`ResamplingFunction` is used by `ResamplerConfig`, which is public, and
`SourceProperties` is used by `ResamplingFunction`, so both should be
public too.

Signed-off-by: Leandro Lucarella <[email protected]>
Using a type alias forces us to use strings to specify types because
Python can't use forward declarations for statements.

Also using a protocol allows for more control, as even when we don't
use it for now, it will also check function argument names if `/` is not
used.

Signed-off-by: Leandro Lucarella <[email protected]>
`Sample` can have `None` as value, but the resampler buffer (and
function) never get any `None` value stored/passed. The reason is saving
a sample with no value provides no extra information and would make the
resampling function implementation more complicated, as it would need
to check values for `None`.

Currently the resampling function will never receive a `None` value but
it still needs to check for them for type-checking reasons.

This commit uses a `tuple[datetime, Quantity]` to store samples in the
buffer and pass it to the resampling function. This way it is trivially
clear that values can't be `None` in this context.

Signed-off-by: Leandro Lucarella <[email protected]>
The `average` function is just a mean function and Python already offers
a function to calculate a mean, so we use that instead.

We also wrap `fmean` to match the `ResamplingFunction` signature inline
so it is clear in the documentation what the default resampling function
does.

Signed-off-by: Leandro Lucarella <[email protected]>
This reduces the benchmark time by about 4x.

Also use `Quantity.zero()` as a shortcut.

Signed-off-by: Leandro Lucarella <[email protected]>
@llucax llucax requested a review from a team as a code owner July 8, 2024 08:24
@github-actions github-actions bot added part:docs Affects the documentation part:tests Affects the unit, integration and performance (benchmarks) tests part:data-pipeline Affects the data pipeline part:actor Affects an actor ot the actors utilities (decorator, etc.) labels Jul 8, 2024
@llucax llucax marked this pull request as draft July 8, 2024 08:25
@llucax llucax added this to the Untriaged milestone Jul 8, 2024
@llucax llucax modified the milestones: Untriaged, post-v1.0 Jul 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
part:actor Affects an actor ot the actors utilities (decorator, etc.) part:data-pipeline Affects the data pipeline part:docs Affects the documentation part:tests Affects the unit, integration and performance (benchmarks) tests
Projects
Status: To do
Development

Successfully merging this pull request may close these issues.

2 participants