[Bug] Occasional Device Disconnects with 4.0.0b9

[DrTom]

One of my custom devices when attached as a PUPDevice gets disconnected by the hub (tested on inventor / spike) occasionally. Such a disconnect happens randomly but likely within 10 minutes with Pybricks 4.0.0.b9. Under Pybricks 3.6.1 the connection seems very stable; two attached devices ran for more than 3 hours without one connection dropped by the hub.

I am experimenting with a several devices which are based on the same hardware and run the same implementation of the UART protocol (LUMP). I am not quite certain what causes the problems but this device tends to send longer packages and those more frequently. The length of the most commonly used mode is 16 bytes. The frequency is slightly higher than 100Hz.

There is also a 32byte mode and it is possible to crank up the frequency further if that helps. I did not yet check if the situation gets worse with those parameters. I would like to help to debug this on the hub but I have no idea how. Since the disconnects are unpredictable and not that frequent it seems instrumentation after the fact is required.

[laurensvalk]

Thanks for opening this issue. It would be nice to get this fixed for the upcoming stable release.

I have just pushed a commit to simplify debugging on SPIKE Prime or Robot Inventor. See pybricks/pybricks-micropython@da722c8

With this firmware, if you connect with https://beta.pybricks.com/ via USB, you will get debug prints. They come from lib/pbio/src/port_lump.c where you will find various debug_pr.

If you can build your own firmware, you can add your own debug_pr to further close in on the issue as you reproduce it. If not, or to get started, you can try installing this ready-made firmware with these steps .

I have intentionally disabled all ports except port B. This way you don't see all the debug prints from the other ports at the same time.

In this version you will see things like this when you pug it in, which is the initial info sequence from the sensor:

set baud: 115200
type id: 48
num_modes: 6
speed: 115200
fw version: 00000004
hw version: 10000000
new_mode: 5
...

And if you unplug the sensor you should see an error condition such as

No data since last keepalive

You may find yourself needing to add debug statements in other files. To do that, simply add

#include <pbio/debug.h>

and then you can call this printf-style function:

pbio_debug("Hello, world %d", 123);

---

Also tagging @ste7anste7an, who had a similar issue with custom devices.

[ste7anste7an]

Thank you for adding these debug possibilities. This will help a lot in getting these problems resolved.

On the other hand, when using UARTRemote to communicate externally, there is no need to emulate the LPF2 protocol. Only, then, the sensor can not (yet) request power through M1 or M2. See the pull request: pybricks/pybricks-micropython#468 If there is anything I can do to get this pull request accepted, I am pleased to do.

[laurensvalk]

We'd still love to know if and where our handling of the LPF2 protocol has changed if that's not too much trouble 🙂

Thanks for your patience on pybricks/pybricks-micropython#468. We would like to look at this before the stable release so that custom device support is "complete".

[ste7anste7an]

I will certainly look at it. As I use code still stemming from EV3 era, I will get rid of the 2400 baud initialization and start immediatly on 115200, as all spike prime LPF2 sensors do. That might already solve some issues. I will post my findings here.

[DrTom]

The keepalive times out:

.time: 87.001  data: (1, -54, 13, 75, -57, 127, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
No data since last keepalive

One thing to note about the device: pushing data is lazy. When the data does not change it will only send data in response of the SYNC request from the hub. So the frequency drops to 10Hz effectively. In 3.6.1 the timeout for missed keepalive must have been 500 to 600 ms, that is what I saw in the logic analyzer. If I am not mistaken it is now 100ms which is very tight with the outlined behavior and the fix would be as simple as this: DrTom/pybricks-micropython@e0c2e55

I wanted to check myself if this works but I am currently having problems to get a build pass on my local environment (MacOS) or on Ubuntu 22.04 within Docker. There is some problem with a BTstack API mismatch. Is this a known problem?

[dlech]

There is some problem with a BTstack API mismatch. Is this a known problem?

No, we build everything on CI, so it should be OK. You probably need to run git submodule sync and git submodule update.

[DrTom]

You probably need to run git submodule sync and git submodule update

Isn't that baked in the makefiles? But I might mix up projects here. Anyways, the build in the container should not be affected by this. What is your build platform? Could there be a mismatch between AMD64 and ARM64 platforms?