Missed beacon observations on 6.0.1 and 5.1 devices

[MadisonBlake]

Expected behavior

Consistent iBeacon observations ranged

Actual behavior

When scanning on a 6.0.1 and 5.1 device at 10Hz with beacons running at 10Hz observations are frequently missed, sometimes for multiple seconds.

This behavior is not observed on 7.1 device (Pixel).

The same code was ran on three devices gathering time and strengths of beacon range entries.
The following is a table summarizing the time difference (in ms) between successive observations

	μ	σ	L∞
Droid Turbo	301	633	10547
MotoE	525	483	4922
Pixel	173	79	560

Further when I test with my own beacon parser (using BluetoothLeScanner.StartScan), I get behavior consistent with 7.1 device on the 6.0.1 device.
To me this indicates it isn't hardware issue.
Maybe I am missing a configuration step somewhere?

Steps to reproduce this behavior

Have an iBeacon within range
set foreground scan period to be 100ms
set foreground between scan period to be 0ms
record beacon as they come into the RangeNotifier object

If needed I can send code to stimulate this issue.

Mobile device model and OS version

Encountered on:

• Droid Turbo 6.0.1
• Moto E 5.1

Android Beacon Library version

2.9.2

IMPORTANT: This forum is reserved for feature requests or reproducible bugs with the library itself. If you need help with using the library with your project, please open a new question on StackOverflow.com.

[MadisonBlake]

I did some investigating and it seems to be a problem with Android when starting/stopping the scanner frequently.

Is there a reason range updates are only done at the end of a scan?
I think it would be generally useful to either allow ranging updates to happen on a higher frequency vs scanning, or have ranging updates happen on any received beacon.

I made some modifications to support that. I will probably drop those changes in a fork sometime soon. I doubt you will want to blindly pull my changes in, but it may be a good to see what I was thinking as useful.

[davidgyoung]

Android 7 did add a restriction that prevents starting and stopping the scanner more than 5 times in a 30 second period. There was a fix put in the library (which is included in 2.9.2) to address this here:

7bfc78e

Are you sure you are testing with 2.9.2? If so this suggest this fix isn't working or maybe there is another root cause. You can see debug messages from the OS if Android is blocking starting scans.

The library is designed to deliver ranging results at the end of a scan cycle so it can average all of the RSSI signals over that period and deliver an aggregate distance estimate. It also simplifies the API for many applications that don't want to get a callback for every packet, but simply want to get callbacks at regular intervals.

[MadisonBlake]

I am getting proper behavior out of 7.0. Improper behavior on 6.0.1/5.1.

[davidgyoung]

I think I understand the issue. I missed from your description that you "set foreground scan period to be 100ms" I think this is the cause of the problem.

Understand that if you set the scan period to such a short interval, you will only detect an advertising packet if the start and end of the packet transmission happens within that 100ms period. (It is also probable that there is some delay before actually starting the scan once commanded, so you probably end up scanning for a period a bit less than 100ms). If a transmitted packet is still being sent when the scanner starts up, or stops, it will be missed. You'll only get a detection if the entirety of the advertisement packet is transmitted during this very short scan period.

For this reason, I do not recommend setting this interval to anything less than 500ms, unless you are transmitting at a very high rate. Otherwise you will miss most packets.

The reason this does not affect 7.0 is because the scan is kept going (not stopped) for 6 seconds at a time in order to comply with Android 7.0's limits on the number of scans you restart.

[MadisonBlake]

That was essentially my conclusion after looking into it more, although I did not know the part about 7.0.
It still interest me that I was missing beacons for multiple seconds at a time. I would think that one packet would of had a chance to be received in that period. It is clear starting/stopping scanner at 10Hz is not appropriate, but I am worried there may be some other mechanism at play.

Overall, I am interested in getting every beacon advertisement as I am applying a different normalization scheme. I would think such a feature would be usable for other scenarios aside my specific application.

To support my specific applications I made changes that expose an optional second cycle that ranges everything mid-scan-cycle. Those changes are in a fork here:
https://github.com/MadisonBlake/android-beacon-library

I am content to keep those changes in that fork. As I said, I believe my scenario may be useful to other people beside me, so feel free to make it a PR or let me know of any improvements/testing that need to be done if you want to pull it into main branch.

[MadisonBlake]

Investigating the ~5 second halt in receiving beacons it seems to be an Android/hardware issue.

First I used 2.9.2 and ran a scan at 1000 ms (1 sec) intervals and saw that I occasionally got 5 second periods of no observations. I then used pure Android functions that android-beacon-library uses (BluetoothLeScanner.StartScan) and got similar 5 second intervals of no observations.

For both it seemed to be around 8 occurrences in 15 minutes, resulting in no data ~3.5% of the time.
the low frequency makes this a minor issues, but something that would be nice to get to the bottom of since my application requires frequent bluetooth updates.

The outages are pretty uniform at 5 seconds. I think android switches the scanning frequency every 5 seconds, so the outages may be related to that?

Again it doesn't seem to be an issue with android-beacon-library, as I was able to reproduce the issue using only Android calls.

[davidgyoung]

Thanks for the update. One idea I had to address the start/stop scanning issue:

The main reason to stop scanning and restart is that on some older devices like the Nexus 4, you only get one scan callback per unique BLE device per scan. To make the library work with these older devices, stopping scans is needed.

But it would be possible to detect if 2+ scan results for the same MAC ever came in for a single sca. if so, we could set a flag indicating this device does not need to stop and restart scanning.

This would help in your case, but only if two packets are ever detected in the scan interval (which may be tough to get to happen with such a short one!)

[davidgyoung]

@MadisonBlake, I made a change in #491 to not restart scans on devices that detect multiple identical advertisements per scan. I believe this may resolve this problem.

[MadisonBlake]

I'll take a look at it sometime this week. I have moved on to other things at the moment, so it may take some time to get back around to this.

Assuming it works, it would be nice for my project to no longer be dependent on my fork where I implemented my own kludge/workaround.

[halindrome]

@davidgyoung question: With the change in #491 if the library decides that the hardware does not need to restart scanning on an Android 7 device, will this effectively work around the 5 scans in 30 seconds limit?

[davidgyoung]

Basically, yes, because you will only be doing one scan. But even before this change, a similar behavior was already implemented to simply not stop and restart scanning (instead leaving scanning on) if it would put you over the Android 7 scan limit.

[davidgyoung]

Closing as this was fixed on #491

[gaj-programmer]

Hi i have a one issue, I am using alt-beacon library, default scanning time 1100 ms, but some time it detects 10 beacon ... and on another scan it detects 6 beacon... it fluctuate a lot, what should i do... to scan all beacons in every scan period. Please Help

[MadisonBlake]

@davidgyoung I apologize for being so late, but I finally got around to looking at this. My kludge was working fine until Android 8.1.0 requires me to pull in the latest android-beacon-library. So I have been looking at how the latest stock android-beacon-library performs for my needs.

I am still seeing some issues with 6.1.0 android devices when running a somewhat fast scan period. My beacon is running at 10Hz and scanning every 300ms. I am seeing the same 5 second halt that I detailed in #473 (comment).

I don't quite remember the conclusion on that issue; if it was something hardware/OS related that we have no control over, or if there was a way to mitigate it.

[davidgyoung]

Sorry that #491 did not help. Based on your comment last May, you said you believed it was a hardware issue.

It may have something to do with channel hopping. Android devices scan on one of three BLE advertisement channels at a time, and switch between them at a device specific rate. Sone Samsung devices, for example, switch about every 5 seconds.

Transmitters are supposed to send each packet on all three channels. But if a beacon does not do this properly it might cause the problem you describe. I would try a different beacon transmitter to see if you get the same results.

See this related discussion of channel hopping and RSSI:

https://stackoverflow.com/questions/49634513/is-there-an-explanation-for-the-regular-oscillation-experienced-in-bluetooth-rss

[MadisonBlake]

I've tried a least two brands of transmitters: On physical beacon, one software simulator on a spare phone.

The hardware's 5 second frequency hopping was my guess too. But the "gaps" seem to occur randomly. Initial analysis indicates that it is random, but I probably need to collect more data to determine. Here is an analysis of a 40 minute run. Plotting the time since last observation shows the 5 second, and occasionally 10, second "gap" occuring.

y axis is ms since last beacon observation. x axis is observation number.

Defining a "failure" as a gap of more than 1000ms the time since last failure is plotted. It looks rather random to me, the histogram of this data (not shown) also indicates randomness.

y axis is "ms since last failure" x axis is the failure number

I also observe similar failures, but much lower rates, when slow the scan period to 2000ms. I was able to reproduce this error outside of this library, so my guess is its more hardware than anything else. But still something nice to solve as I am guessing this issue is widespread across all 6.1.0 devices.

I wonder if something funky is happening if a scan is started exactly when the OS/Hardware is changing frequency? Like it's trying to scan on one channel, but the hardware is operating in another? I don't know, just and idea.

I will be happy to share my testing methodology and results. It might be too much for github issues. Feel free to email me. Or I can share it here.

Also for what it's worth. My testing on a Nexus 6, Pixel XL, and Pixel 2 XL are perfect.