As many of you will be aware, work has been underway updating the SensorGnome software to ensure compatibility with newer Raspberry Pi models, introduce some new features, as well as bring the security and interface into line with what’s expected in 2022. This process has been largely led by Thorsten von Eicken, who has given a few updates on progress here previously.
We’re getting close to being able to release this more broadly to the community, but still have some testing and refining to do. In order to improve and speed up this process, we’re hoping to enlist a few additional testers from the community to run the newest images and provide feedback.
Ideally these would be people with a solid background with Motus itself, familiarity with the previous SensorGnome software and with the Raspberry Pi in general, and of course the time and interest to do some testing and troubleshooting on the side.
If you’re interested in being part of the testing, please let us know.
I’d be happy to test out the new software. I’ve been working with automated telemetry and motus since its inception and have used pretty much every version of sensorgnome so I’m familiar with the system. I can implement the new software here in Ithaca on two towers that I have setup for bird club here and can deploy new version of the software on receivers this winter at our long-term study site in Jamaica (lots of active tags).
Only a recent Motus participant but already have 2 x RPi 3B stations on test, and a RPi 4B with the sg-armv7-rpi-bullseye-2022-035 release also on test. Happy to volunteer to assist with testing new software.
FYI now retired but with a 20+ years career in technology. Everything from programming, to systems management but specialising in network communications prior to retirement. Currently a technology support volunteer with local wildlife charities.
Thank you for the replies! A couple of users have installed a recent version of the new sensorgnome software and I have a bunch of action items from that. I have also been setting up remote monitoring and logging for the new SGs which should make it easier for me to troubleshoot issues that come up. I’m hoping to have a new build worth testing in a few days and will post here at that point.
Dear volunteer testers :-), a “release candidate” for the new Sensorgnome software is available! As far as I can tell, most everything is working but there are a few known issues listed below and the configuration still has a number of rough spots. The primary purposes of the release candidate are to discover issues with the set-up process and to ensure that the data path from radio capture to motus server upload is robust.
Prerequisites:
Raspberry Pi4B, Raspberry Pi3B, or Raspberry Pi Zero-2W
ideally some radio (FCD, CTT, or RTLSDR) and corresponding test tag
local web interface for configuration and monitoring
download data direct from web interface
security upgrade: current OS and all access is secured (HTTPS, SSH, or WPA2 for the hot-spot)
concurrent WiFi client and hot-spot support
direct upload to Motus servers via HTTPS, no more SSH tunneling
remote monitoring and upcoming remote management
same data path as “old” Sensorgnome releases: the software to process radio data has not changed
Known issues:
The upload authentication is freshly rewritten but still in flux as it needs some Motus server back-end improvements, it does work and the docs have more details
The Lotek receiver frequency cannot be set easily (defaults to 166.38Mhz), one has to manually edit the acquisition.txt file via the command line (SSH).
Unplugging an RTSDR radio from a USB port is not detected by the software, a reboot is necessary to “clean things up”.
Editing the Sensorgnome “short label” which appears in data filenames in the Web UI has no effect.
Remote monitoring and management via a Sensorgnome server is only partially implemented.
The HTTPS certificate is not currently auto-renewed, it expires in December, a software upgrade will be needed to renew it.
The formatting of help text in the Web UI’s “i” buttons is poor, especially in the radio detections log explanation.
There has been relatively little concern for internet bandwidth so operation on cellular is not recommended on low-data plans
Recent changes (some people have tested earlier versions, thank you!):
the upload authentication has changed, you no longer have to enter creds (but can)
the image uses the “stable” package repository for upgrades to allow for more controlled upgrades
support for hotspot-less initial installation
initial implementation of remote monitoring If you try out the new release, please report back to the mailing both good and bad!
I’ve been chugging along and added web UI support for selecting the Lotek radio frequency (166.38/150.1/150.5Mhz) and also fixed the plugging/unplugging of RTLSDR radios. The new “RC5” image can be found at: https://sensorgnome.s3.amazonaws.com/images/sg-armv7-rpi-2.0-rc5.zip
If you have flashed the previous version you can upgrade using the “software” tab, just be aware that it takes patience and looking for progress in the “upgrade log” widget.
Thorsten
Two people reported a problem this morning with setting the station “short-label” and indeed, embarrassingly, it doesn’t work. I must be sleep-walking through the testing, done it too many times at this point…
rtlsdr detection race condition on rpi4 at boot time
issue with upload session token needed a refresh on certain errors
If you have flashed a previous RC you can upgrade using the “software” tab, just be aware that it takes patience and looking for progress in the “upgrade log” widget. After the upgrade, you can edit the label in the Web UI, the label at the top-left of the title bar only changes after a page reload, though.
Thank you Michael and Edwin for testing RC5, I really appreciate it!
Thorsten
NB: if you find issues or have questions please feel free to reply-all, this way others can avoid running into the same issues… If you’re getting too many emails, switch the group to once a day digest emails, or something like that. (I wish this was a proper forum…)
Edwin wanted to connect a generic GPS module via the serial port to his Sensorgnome and while it’s simple there are many blind paths when it comes to Raspberry Pi serial ports… I wrote up an FAQ entry which hopefully will make it easier for the next person: Initial software installation | Sensorgnome V2 User Guide
As always, if you find a mistake there or can suggest an improvement please let me know so I can fix it!
“Upload failed while uploading with error: Error: Invalid upload date in info: [object Object] at DataFiles.updateUpDownDate (/opt/sensorgnome/control/datafiles.js:373:23) at MotusUploader.doUpload (/opt/sensorgnome/control/motus_up.js:292:23) at processTicksAndRejections (internal/process/task_queues.js:95:5) at async MotusUploader.doUploadAll (/opt/sensorgnome/control/motus_up.js:242:16)”
Michael, thanks for testing and thanks for reporting! This is an interesting one:
the uploads are actually functioning, so your SG is operating and uploading new data, that’s the good news
the less good news is that it’s failing to upload data files created before it got the current time and that causes the error
A bit of background: the SG generally uploads data files when a rotation to the next set of files occurs, which is typ once an hour. It then uploads files from today that haven’t yet been uploaded. Then, if there are older files that haven’t been uploaded it picks one day with such files and uploads those files. In your case there are files for 2022-09-14 which is the date the SG initializes with the first time it’s booted before it gets the current time. When it tries to upload those files the Motus server responds with an error stating that those files have already been uploaded on 2019-09-14 (note the year) and a date sanity check in the SG then raises the “invalid upload date” error you see. Soooo… the obvious question is why the server is responding with this “already uploaded” error. In addition, I need to think what to do with this persistent error message, although, at some level it’s good 'cause I would not have found out about this without it…
WRT your SG:
I would appreciate if you could keep it running 'til I resolve the issue. If you can test other aspects that would be great, I see it’s a Zero-2W which I have only briefly tested.
Could you use the “download all” button in the web UI and email me the archive you get? This may tell me why there is a hash collision leading to the “already uploaded” error and how to prevent that.
Thank you for all the work you are doing on the new Sensognome software. I really like the new user interface.
I got a station set up on November 8th running on a RPI 4B. Several days ago it hung up while uploading to the motus server. After several hours of showing “uploading” in the upload result box, I wound up rebooting the Sensorgnome through the UI. All things seemed to work after that. Do you need or want log information? Do you have a way of monitoring our systems yourself?
I have a couple of questions:
I have not installed a momentary switch on my Adafruit GPS yet. Does the new software function like the old with a momentary switch installed (double-press for WIFI, three second press for shutdown)?
I don’t see my station on the Sensorgnome status page. Is there somewhere where we can see the status and maybe log in for remote administration? Or, is this yet to be implemented?
Thanks for checking out the new version and providing feedback!!!
Can you provide the Sensorgnome ID and the approx time-frame so I can look into it? I do have central monitoring set-up so I can troubleshoot. The web site is not quite ready to be made available as some stuff is too awkward to explain […].
Looking at the monitoring site, the following plot shows the upload status of a bunch of new SGs. (Ignore SG-68B4… which has a registration issue.) Where you see red bars on all SGs it’s a server-side issue, incl. 24 hours on 11/16-11/17. I’m not seeing an issue specific to your SG so it would help me if you could also tell me the approx time-frame of the problem you encountered.
Thorsten
Thanks for looking into this. According to the “Booted” box it looks like I rebooted it on November 18th around 22:00 UTC. If I remember correctly it hung up some time on the 17th. I figured it was a server thing. Should I have left the sensorgnome on to sort it out? I had given it close to 24 hours to upload correctly. I wish I had taken some snapshots of the UI before rebooting it.
Levi, there definitely was an issue and your SG stopped uploading, most likely due to an uncaught error. I’m still looking into it to try and figure out what might have happened and how to mitigate the issue. Thanks for bringing this to my attention!
Thorsten
Thorsten, FYI my RPiZ2W test station has now been running for 4+ weeks, uploading quite happily albeit just GPS co-ords. With just an omni antenna and being in the middle of the Island unless a tagged bird/bat takes a short curt through the central valley then I don’t really expect to “see” a tag.
There’s plans to deploy “real” stations on the Island in the new year so can I please ask if there’s any notion of when the software we have on test will become the standard distribution? It is just so we know which hardware/software to deploy. Thank you. Mike
Michael, thanks much for the positive feedback! If you have a test tag it could be helpful if you could place it near the station, even if just for a couple of hours.
Regarding “standard distribution”, I can’t make any statement for Motus in that respect. What I can tell you is that I’m definitely committed to supporting the software. I have a to-do list that I hope to work on shortly but none of the items are really show-stoppers. Top item is making the central monitoring site accessible to anyone. The only thing I would be skittish about is having non-connected stations just because there’s not enough reliability history for having unconnected stations out there: I would feel really awful if someone reported back that the station they had out there for months didn’t collect anything 'cause it crashed a couple of days after install!
Let me ask you: will the stations have internet access (I can work with low volume cellular), will it be possible for personnel to check stations occasionally if there is reason for concern, and what is your rough timeline in terms of station count and install date? And if stations won’t have internet is that due to lack of cell coverage or due to other reasons?
Unfortunately I don’t have a test tag in the immediate. Hopefully we may get access to one in the near future.
Timeline: installations are scheduled to begin from March 2023 onwards.
I have accumulated a small stock of RPi3B on which we can run the current “standard” software if necessary to be upgraded later.
The intention is to have all the stations connected to the internet whether by courtesy of the station site owners, or cellular.
Yet to be confirmed with the project team, the intention would be to have all SGs connected to the VPN server to enable remote access.
For now I have installed CACTI network monitoring on the VPN server which has been configured to simply ping the SGs giving us an up/down indication.
That sounds like a great set-up and pretty ideal!
I did some more work on the monitoring site and will get you access within a few days. You can then decide whether you need the cacti set-up at all.
I think the ideal plan would be:
set-up the HW for a typical station configuration on the bench (i.e. minus antennas and field install)
place a test tag nearby
run this 'til comfortable with performance
perform tests like reboot, power outage, etc
perhaps set-up a second HW and load old software for A/B comparison
I don’t know whether you have the resources to pull this off, if not, maybe we should have a huddle with Motus folks to figure something out…