Created
April 10, 2019 22:51
-
-
Save christianromney/e0beade6aaf032b6e71840f8f772ea7c to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Hi Christian, | |
Over the past few days, our reliability has not met our standard | |
of excellence. We are writing to apologize for any disruptions | |
this may have caused for you and your work. Resolving these | |
issues is our team’s top priority, and we are doing everything we | |
can to resume the level of service you expect from CircleCI. | |
What happened: | |
The majority of recent incidents are related to an unexpectedly | |
high increase of jobs processed on our platform. This new level | |
of requests has caused one of our primary datastores to be much | |
slower than normal, and at times, unresponsive. These datastore | |
issues have resulted in slow or stopped job processing for some | |
customers. | |
What we are doing to resolve these ongoing issues: | |
Historically, we have been able to plan and support the ongoing | |
growth of our platform through predictable scaling of this | |
datastore. In this instance, however, we have witnessed 10-100x | |
increases in latency and complete datastore stalls. We are | |
working closely with the datastore technology provider to tune | |
the datastore itself and adapt our own application code to make | |
optimal use of the datastore. | |
Where to learn more: | |
We will release a full root cause analysis when we have observed | |
a sufficient period without further datastore incidents. When we | |
do so, you will be able to find it on our Status Page, and we | |
will also link to it on Discuss here. | |
Thank you for your continued patience and support. We know that | |
your CI/CD pipelines are a mission-critical part of the work your | |
team does. We are committed to being as transparent as we can, | |
and apologize again for any impacts you and your team have felt. | |
Sincerely, | |
The CircleCI Team |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment