Zooniverse - Notice history

All systems operational

Operational

Project Pages and Classifications - Operational

Project Builder - Operational

Talk - Operational

Caesar - Operational

Classrooms - Operational

Notice history

Apr 2023

No notices reported this month

Mar 2023

No notices reported this month

Feb 2023

Stats Service Down / Degraded
  • Resolved
    Resolved

    The Zooniverse team is excited to announce that we've launched our new backend stats service, ERAS, and project + organization stats pages have returned to speedy working order. Thank you for your patience while we built and deployed this new service!

  • Update
    Update

    Our dev team continues their work on a new stats service to replace the current, permanently degraded service. Work on the new service is nearing completion, with roll-out of the new service taking place by the end of October.

    A reminder: project stats via the current degraded service will often take multiple minutes to show, and is only successful a fraction (<50%) of the time. That said, stats can still be obtained in many cases with a little perseverance and page reloads. We thank you for your continued patience and look forward to the new service in the coming weeks.

  • Monitoring
    Monitoring

    After working with our cloud computing vendor for multiple weeks, we were unable to restore the stats service to its expected performance. While we were able to bring a degraded version of the service back online, we expect long query times, request timeouts, and overall poor performance from this service for the near future.

    In response, our dev team has started building a new stats service to replace the older, permanently degraded service. Work has begun, and we aim to bring the new service online by September 2023.

    We thank you for your continued patience.

  • Update
    Update

    We are continuing to make slow progress in restoring the stats service. On Thur 6 April, we were able to revive the service, but it is operating in a degraded state (requests are running slow, plots on project stats page will take up to a minute each to populate, etc). The Zooniverse development team continues to work with our cloud computing vendor to improve the performance, and we are beginning to take longer term steps (over next few months) to replace the problematic backend service.

  • Update
    Update

    We continue to work with our cloud computing vendor to resolve the issue that continues to affect our stats service. Again, we thank you for your continued patience.

  • Identified
    Identified

    Investigations indicate that the cause of the ongoing stats service outage is due to an issue with a cloud-hosted backend service. We are working with our cloud computing vendor to pinpoint and resolve the issue; thank you for your patience as we work to bring the stats service back online

  • Investigating
    Investigating

    Starting at 10am US Central time on Monday, February 27, the Zooniverse stats service became non-responsive. This service provides histograms of classification and comments on a project stats page, which are not functioning at this time. We are actively investigating the issue and will provide updates here as we learn more.

    Please know that this issue does not affect data collection in any way: classification submitted to project workflows and comments submitted to Talk are all functioning normally. Furthermore, project level stats that appear on the project landing page, individual stats on personalized Zooniverse homepage, and individual stats on the new FEM version of the project classify page all remain functional.

Feb 2023 to Apr 2023

Next