Corvic - Errors logging into app – Incident details

All systems operational

Errors logging into app

Resolved
Operational
Started 6 months agoLasted about 4 hours

Affected

App

Major outage from 6:00 PM to 8:31 PM, Partial outage from 8:31 PM to 9:01 PM, Operational from 9:01 PM to 9:31 PM

Updates
  • Postmortem
    Postmortem

    Mini incident report:

    An incident in GCP [1] caused two issues. First, it affected CloudFlare [2] which affected Corvic's identity provider and prevented users from logging into Corvic site-wide. Second, Corvic deployments based in GCP were directly affected. The downtime for these deployments was particularly severe as the us-central1 region in GCP took the longest to recover.

    [1] https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

    [2] https://www.cloudflarestatus.com/incidents/25r9t0vz99rp

  • Resolved
    Resolved
    This incident has been resolved.
  • Update
    Update
    GCP us-central1 recovery is ongoing and expected to complete in the hour.
  • Update
    Update

    We are still monitoring recovery in GCP us-central1.

  • Update
    Update

    The upstream cloud provider has applied a fix, and we are seeing services recover.

  • Monitoring
    Monitoring

    We've identified that a cloud provider has been having a major outage. We are monitoring the recovery of their systems.

  • Identified
    Identified
    We are continuing to work on a fix for this incident.
  • Investigating
    Investigating
    We are currently investigating this incident.