Resolved -
All nightly UDP ingests relying on Canvas data have no successfully ran with fresh Canvas data.
Oct 22, 14:32 UTC
Update -
Instructure has reported that all Canvas systems are operational. We have been monitoring the caliper feeds and are observing event ingest rates typical of normal operations.
Canvas data imports continue to be delayed but should be fully refreshed by Wednesday morning (10/22).
Oct 21, 16:05 UTC
Update -
Instructure has reported that Canvas functionality has been restored and they are monitoring for stability. Unizin is seeing caliper events from Canvas flow into the UDP at a rate comparable to rates pre-incident. We will continue to monitor the endpoint.
You can follow Instructure's status of the issue at https://status.instructure.com/
Oct 21, 00:38 UTC
Monitoring -
We are starting to see Canvas caliper events begin to flow. We will continue to monitor our endpoint and the event volume while the underlying AWS issue is fully resolved.
AWS has reported the following:
Our mitigations to resolve launch failures for new EC2 instances are progressing and the internal subsystems of EC2 are now showing early signs of recovering in a few Availability Zones (AZs) in the US-EAST-1 Region. We are applying mitigations to the remaining AZs at which point we expect launch errors and network connectivity issues to subside.
You can follow the status of the issue at https://health.aws.amazon.com/health/status and https://status.instructure.com/
Oct 20, 17:53 UTC