Problem with Pi3B and Bullseye

[JonM]

I had a Pi3B working with an older Raspian release.  When the SD card died, I was forced to update to the most recent "Bulleye" release (Debian 11, kernel 5.15.33)  Now I'm getting binary data from /dev/serial0, instead of NMEA sentences.

I'm at a loss whether this is a software problem, or a configuration problem with the BerryGPS-IMU

[JonM]

More information; looks like something broke with newer Raspian releases.  I went back to Raspian 8 (Jesse) and the first thing I noticed was the dmesg output is different:

ttyS0 at MMIO 0x0 (irq=220, base_baud=31250000) is a 16550

If I recall, with Raspian 11, the base_baud was 50000000

I'm happy knowing the hardware is OK, but what is my path forward with newer Raspian releases?

[Mark Williams]

do you see NMEA sentences if you kill GPS and run gpsmon manually?
kill gps;
sudo killall gpsd

run it manually;
gpsmon /dev/serial0

Mark --OzzMaker.com --

[JonM]

I've just been testing by running "cat /dev/serial0".  No reason to start gpsd if I can't see the NMEA sentences coming across the serial port!

I just downloaded Raspian 9 Stretch, with the 4.14.98 kernel and it works correctly.

Similar dmesg output:

ttyS0 at MMIO 0x0 (irq=166, base_baud=31250000) is a 16550

My conclusion is that something changed between Stretch and Buster.

[JonM]

Since updates are still available for Stretch, I updated to the 4.19.66 kernel, and can confirm that reading from /dev/serial0 still works, and I was able to install and configure gpsd too.

[JonM]

More progress... ran rpi-update to update the firmware.  This was successful, and dmesg reported

ttyS0 at MMIO 0x3f215040 (irq=86, base_baud=50000000) is a 16550

This is same base_baud that was reported by Raspian 11 when it wasn't working.  The update also updated the kernel to 5.15.33.  Since cat /dev/serial0 was working, I went ahead and installed gpsd and gpsd-clients; as expected those worked too.

I went back to Raspian 11 with 5.15.32 (lower minor number?)

ttyS0 at MMIO 0x3f215040 (irq=71, base_baud=50000000) is a 16550

AND IT WORKED!  Sorry for yelling, but this is exciting!

Tempting fate, I ran rpi-update again, and ended up with the 5.15.34 kernel...

The GPS is still working!

If I had to guess, it was the firmware update that was needed to correspond to the 5.x series kernel.

[JonM]

The problem is probably not solved... there's been no software, firmware or operating system changes, and I'm back to having problems reading from the BerryIMU-GPS card.   I'll get a valid event from time to time, but not the once-per-second GPS report that I'm expecting.

[JonM]

Definitely leaning toward a hardware failure... I have a second nearly identical device that is working properly.  Its a Pi4, not a Pi3; otherwise software is identical.   Not easy to swap the BerryIMU-GPS cards to see if the problem follows the card.  Any other ways to try to diagnose?

[Mark Williams]

You can try and see what the difference is using stty -F /dev/serial0 -a.
You can also test using a console loopback, connect TX pin to RX pin on the Pi and type something in a console program, it should echo back to you.

Is this correct, the GPS works on an old OS but not a new OS on the same Pi?

Mark --OzzMaker.com --

[JonM]

Thanks Mark-

My original thought was that it was a problem with firmware and OS level, and that I cleared that up when I upgraded the Pi3B.  Then the problem came back.  The same firmware and OS are running on another Pi4, I just don't have physical access to it right now.  I'm going to try to find another Pi and swap the BerryIMU-GPS to that and see if the problem follows.  I'm also going to look and see if one of the solder joints might have been compromised.

[JonM]

I moved the BerryIMU-GPS to an older RPi2.. (running Raspian 8 - Jessie, with up to date firmware)

$ stty -F /dev/ttyAMA0
speed 9600 baud; line = 0;
-brkint -imaxbel

dmesg

3f201000.serial: ttyAMA0 at MMIO 0x3f201000 (irq = 114, base_baud = 0) is a PL011 rev2

and if I cat the serial port, I get the following:

Does any of this make sense:

$GNRMC,001817.00,V,,,,,,,030522,,,N*6A

$GNVTG,,,,,,,,,N*2E

$GNGGA,001817.00,,,,,0,00,99.99,,,,,,*77

$GNGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*2E

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,00,txbuf alloc*61

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

$GNTXT,01,01,01,NMEA unknown msg*46

When moved back to the Pi3, dmesg:

3f215040.serial: ttyS0 at MMIO 0x3f215040 (irq = 71, base_baud = 50000000) is a 16550

$ stty -F /dev/serial0
speed 9600 baud; line = 0;
-brkint -imaxbel

and if I cat the port, I get garbage...

$ cat /dev/serial0
�b\('j
3�
�;pնn�Nb��� �\
t���J#Xq�b5�('j
-w�� �
$�'
#7� $"�0 "3=��!+E2 ^ٵb('j
�HUX��'?�� n
�b('j
^B��T���b('j
���� X?��b ('j
����)��ba('j

 

I need to find some wires to try a loopback test

[Mark Williams]

FYI, this is what I get on a Pi Zero i just tested. Doesn't look different
[ 2.963607] 3f215040.serial: ttyS0 at MMIO 0x3f215040 (irq = 86, base_baud = 50000000) is a 16550

pi@raspberrypi:~ $  stty -F /dev/serial0
speed 9600 baud; line = 0;
-brkint -imaxbel

Try the stty command with -a stty -F /dev/serial0 -a so you get all values, and see if this is the same between your Pis.
The Rpi2 looks like it is working
$GNTXT,01,01,01,NMEA unknown msg*46
The above message is the GPS telling you that it is seeing data sent from the Pi to the GPS it and doesn't understand them. The Pi is most likely echoing back NMEA sentences.. or the console is still enabled on the serial. You can disable echo using stty -F /dev/ttyAMA0 -echo

Mark --OzzMaker.com --

[JonM]

Is there any chance that the ublox has a binary protocol, and somehow it's switched to that?

There isn't a getty running on the port.  This morning I noticed that even with binary data on the port, gpsd seems to be working.

SOLVED.... I just ran "gpsctl -n" and it's back to human readable NMEA sentences.

Have to wonder what caused the GPS receiver to switch into binary mode?

Thanks for the help!

[richardp]

yes it sometimes will if using gpsd.
This is why I think Mark asked you to kill gps and start it manually using gpsmon /dev/serial. Trying it this way would force it to use NMEA by default.

[JonM]

I'll keep that in mind, but I think when I tried running gpsmon before it didn't have any effect.  I should have been tracking the version of gpsd, instead of the OS version, since that seems more important now.

I may start using the "--readonly" flag, so the gpsd doesn't try to modify the device.

[Author]

Posts