We sincerely appreciate your patience over the last few weeks. Since our last message, we have replaced hardware and rebuilt software that were contributing to data processing delays. The backlog of data that had built up has now been processed, and both manual and automatic uploads resumed last week. If you have been waiting to upload data to the Motus website, you can now do so.
However, the growth of Motus in recent years has also meant an increase in data volume, so we are continuing to monitor the processing queue to ensure it keeps operating properly. We are working on major updates that will significantly improve system performance and handling of large amounts of data, but those will take some more time to implement. We will provide another update if any further issues arise in the short term, and as implementation of system upgrades progresses.
Once again, we sincerely thank you for your patience during these delays and we apologize for the inconvenience.
Could you just confirm the current status of data processing? It would appear that since this message was posted, data processing continues to be backlogged, with the stations in our project not updating since 2/3. Should we anticipate similar backlogs to occur throughout the next few weeks and months? Any updates greatly appreciated.
This appears to be a separate issue associated with sensorstations - data processing is working for connected sensorgnomes, which are up-to-date, but you’re correct that data from sensorstations hasn’t been processed since Feb 3. It seems that there is an issue preventing data coming from CTT reaching the Motus database - I checked my sensorstations on the CTT web portal, and the data is current there. We’re working on this issue, and once fixed, the data will catch up, but I appreciate this is frustrating particularly given the other recent data processing issues. I’m sorry for the inconvenience, and I’ll post an update as soon as possible.
Just an update to note that the issue with sensorstation data has been fixed and the data are coming through to the Motus database now. It may take a few days for all the data to catch up. My apologies again for the inconvenience.
It appears all 166 data is currently backed up to ~ March 23/24th based on antenna activity data. Which is also causing wonky “latest data” dates across all stations, despite them appearing connected either via CTT website or sensorgnome.net. Is this 2+ week delay going to be the norm going forward, or is there still updates happening and a data push to (relatively) catch it up going to happen again? Mostly wanting to know what to expect going forward. Thanks!
Data uploading seems unstable for the moment, and I cannot manually upload anything either (MOTUS web page says “ERROR”). Any particular reason, and any forecast on a solution to the problems?
Hi Garrett, I’m so sorry for the delayed reply. We’ve been working on rectifying the data backlogs, but unfortunately it has been going more slowly than hoped. So yes, I would say that a ~2 week backlog is likely to persist for sometime yet. We are working on a solution, but I don’t expect it will be implemented in the near future. We will be posting an update to the entire community soon. I’m sorry I don’t have better news and thanks so much for your patience.
Hi Arne, yes, we are continuing to experience a data processing backlog and periods of time where manual uploads haven’t been working are related to that. My apologies, I know it’s frustrating when uploads work only intermittently. We are going to start pausing uploads on a schedule with hopes that this can help resolve some of the backlog, and while not entirely fixing the issue, at least allow folks to plan their upload times around those scheduled pauses so it’s more predictable when manual uploads won’t work. Going forward the ability to upload will be paused between 8 pm and 8 am Eastern Daylight Time, so I would advise uploading outside of those hours. We will post a message with more details to the entire community shortly. Thank you for your patience.