Infinity Bots - Urgent CPU error – Incident details

All systems operational

Urgent CPU error

Resolved
Major outage
Started almost 2 years agoLasted 1 day

Affected

Databases

Major outage from 3:42 PM to 4:52 AM

Olympus

Major outage from 3:42 PM to 4:52 AM

Popplio

Major outage from 3:42 PM to 4:52 AM

Client

Major outage from 3:42 PM to 4:52 AM

Reedhooks

Major outage from 3:42 PM to 4:52 AM

Vanity Links

Major outage from 3:42 PM to 4:52 AM

Updates
  • Resolved
    Resolved

    All services and data have been moved to a better/more powerful server and have been back online for almost 24 hours with no further signs of issues or failures. These issues seemed to be caused by a hardware component or kernel on the previous server reaching the end of its lifespan.

    We will continue to monitor this issue but at this time services have been restored!

  • Identified
    Identified

    We have begun moving our data to a new more powerful droplet

  • Investigating
    Investigating

    All systems are once again failing and we are working with our providers to get this sorted!

  • Monitoring
    Monitoring

    We have identified this as failures within our kernels causing the server to hang unexpectedly, we have since rolled out a fix to most of our services and will continue to investigate for a permanent solution.

    At this time services have been restored but please be prepared for degraded or unreliable performance!

  • Identified
    Identified

    An urgent CPU lockup error has been found and is being looked into, during this time, all services on IBL may be down or hang for extended periods of time