The bias computation is deterministic, so it should match exactly, but it does not. Its very close, and is likely simply a difference in parameters, but its worth understanding why its not exact in case there is a deeper misunderstanding
In the following plot, I'm plotting the bias computed by the refactor acquisition code, as well as bias computed post-hoc by our analysis tools. The analysis tool computation matches exactly with how the v1 implementation acquisition code computed bias.
- Note the v1 version computes the bias every 10 trials and then backfills. So what we are looking for is the right-side of each v1 value to match the refactor bias, but it does not match.
- key parameters:
- n_trials_back = 5
- max_window = 200
- cv = 1
- compute_every = 10 # this is 1 for the refactor, and that should be fine
The bias computation is deterministic, so it should match exactly, but it does not. Its very close, and is likely simply a difference in parameters, but its worth understanding why its not exact in case there is a deeper misunderstanding
In the following plot, I'm plotting the bias computed by the refactor acquisition code, as well as bias computed post-hoc by our analysis tools. The analysis tool computation matches exactly with how the v1 implementation acquisition code computed bias.