Major Cloud Providers – Monthly Outage Recap July

MS Azure

7/30/19 RCA – Issues logging in to the Azure Portal with a Microsoft AccountBetween 21:00 UTC and 22:24 UTC on 30 Jul 2019, a subset of customers may have experienced intermittent error messages and failures when logging in to the Azure Portal with a Microsoft account. This issue affected a subset of MSA users who could not authenticate or manage their accounts.

DownDetector

7/17/19 Problems at Microsoft Azure

Microsoft Azure is having issues since 7:13 PM EDT Most reported problems:

  • Website hosting (51%)
  • Cloud services (42%)
  • Virtual machines (6%)

7/8/19 Azure Active Directory – Password ChangesBetween 18:09 and 22:32 UTC on 08 Jul 2019, a subset of customers using Azure Active Directory may have experienced password change issues. For hybrid customers, passwords would have appeared to have changed, but the password change with AAD would have failed. For cloud only customers, password changes with AAD would have resulted in failures.

7/4/19 RCA – Connectivity Issues – UK SouthBetween 06:18 UTC and 16:25 UTC on 04 July 2019, a subset of customers leveraging Storage in UK South may have experienced service availability issues. In addition, resources with dependencies on Storage may also have experienced downstream impact in the form of availability issues.

7/3/19 Azure Monitoring (Diagnostic logs, Autoscale, Classic Alerts (v2))Between 02:00 and 08:00 UTC on 03 Jul 2019, a subset of Azure monitor customers may have received failure errors while performing service management operations – such as create, update, delete – for Autoscale settings, Classic alerts and Diagnostics settings.

7/2/19 Azure Services – Intermittent Service Availability IssuesBetween 19:20 and 22:20 UTC on 02 Jul 2019, a subset of customers using Microsoft Azure Services may have intermittently experienced degraded performance, latency, network drops or time outs when accessing Azure resources due to a network event. This impact would have potentially spanned multiple Azure services.

-DownDetector

Problems at Microsoft Azure

Microsoft Azure is having issues since 3:37 PM EDT. Most reported problems:

  • Cloud services (50%)
  • Website hosting (37%)
  • Virtual machines (12%)

AWS

7/30/19 Capital One breach occurring March 22-23, 2019 has come to light. In one of the biggest data breaches ever, a hacker gained access to more than 100 million Capital One customers’ accounts and credit card applications as far back as 2005. 

7/3/19 VMware Cloud on AWS Console availability issue Users may experience issues accessing the VMware Cloud on AWS Console. 

Start Time: July 03, 2019 05:55 PM UTC 
End Time: July 03, 2019 07:20 PM UTC

-DownDetector

7/2/19 Problems at Amazon Web Services

Amazon Web Services is having issues since 10:20 AM EDT. Most reported problems:

  • EC2 (66%)
  • AWS Console (19%)
  • S3 (14%)

Other Platforms

7/29/19 VMware Cloud Services Availability Issues Language localization for all Cloud Services UI content is unavailable.

Start Time: July 29, 2019 14:15 UTC
End Time: July 29, 2019 14:45 UTC

7/27/19 SAP Cloud Platform KSA (Riyadh) [neo-sa1] – Service Advisory Since approximately 07:38 – 07:57 UTC A general disruption is impacting the availability of applications and services

7/26/19 VMware Cloud on AWS – SDDC provisioning failures New SDDC provisioning is not working. 

Start Time : 26 July 2019 02:50 AM UTC 
End time : 26 July 2019 04:07 AM UTC

7/25/19

-Google Cloud Console Incident #19007 Cloud Console Dashboard Errors. Incident began at 05:13 and ended at 19:59 (US/Pacific) lasting 14 hours 46 minutes

-Google Cloud Composer Incident #19001 We are investigating errors with creating and deleting Cloud Composer environments. Incident began at 05:13 and ended at 19:59 (US/Pacific) lasting 14 hours 46 minutes

-Google Cloud SQL Incident #19003 Customers are not able to create new private IP instances in Cloud SQL.  Incident began at 05:13 and ended at 19:59 (US/Pacific) lasting 14 hours 45 minutes

-Google Cloud Functions Incident #19005 We are experiencing elevated deployment errors for projects which are deploying to GCF for the first time. Incident began at 05:13 and ended at 19:59 (US/Pacific) lasting 14 hours 45 minutes

-Google Cloud Datastore Incident #19004 Customers are not able to create new databases in Cloud Datastore.Incident began at 05:13 and ended at 19:59 (US/Pacific) lasting 14 hours 46 minutes

-Google Cloud Firestore Incident #19002 Customers are not able to create new databases in Cloud Firestore. Incident began at 05:13 and ended at 19:59 (US/Pacific) lasting 14 hours 45 minutes

7/24/19

-SAP Cloud Platform KSA (Riyadh) [neo-sa1] – Service Advisory Since approximately 09:04 – 09:46 UTC UI Theme Designer is unavailable

-SAP Cloud Platform US East (Sterling) [neo-us3] – Service Advisory Since approximately 04:57 – 05:09 UTC A general disruption is impacting the availability of applications and services

-SAP Cloud Platform Brazil (São Paulo) [cf-br10] – Service Advisory Since approximately 00:51 – 02:34 UTC Applications that are using the Secure Store service might not be able to perform certain operations like storing and retrieving credentials, signing and verifying of digital signatures, encrypting of and decrypting of messages.

7/21/19 VMware Cloud Services Console Login availability issue Users would experience intermittent issues accessing the VMware Cloud Services Console

Start Time: July 21, 2019 00:05 UTC
End Time: July 21, 2019 04:05 UTC

7/19/19 SAP Cloud Platform US East (Sterling) [neo-us3] – Service Advisory Since approximately 03:30 – 05:23 UTC Lifecycle management operations for Java applications cannot be executed. Virtual Machines may be inaccessible

7/18/19

-SAP Cloud Platform US East (Sterling) [neo-us3] – Service Advisory Since approximately 07:15 UTC – 18:19 UTC Some of the Services could be intermittently unavailable

-SAP Cloud Platform China (Shanghai) [neo-cn1] – Service Advisory Since approximately 11:00 UTC until 11:36 UTC Connectivity and connections established with Connectivity are unavailable

-SAP Cloud Platform Europe (Rot) – Trial [neo-eu1-trial] – Service Advisory Since approximately 09:40 until 10:06 UTC HTML5 applications are unavailable

7/17/19

– SAP Cloud Platform Europe (Rot) [neo-eu1] – Service Advisory Since approximately 07:25 until 08:39 UTC A general disruption is impacting the availability of applications and services

– SAP Cloud Platform US East (Sterling) [neo-us3] – Service Advisory Since approximately 08:44 until 20:40 UTC Creation and deletion of new databases is impacted.

– SAP Cloud Platform Japan (Tokyo) [neo-jp1] – Service Advisory Since approximately 20:48 UTC until 22:15 UTC Lifecycle management operations for Java applications cannot be executed

7/15/19

-Google Cloud Storage Incident #19005 Issues with Google Cloud Storage Object Lifecycle Management on US multi-regional buckets. Incident began at 10:24 and ended at 12:15 (US/Pacific) lasting 1 hour 51 minutes

SAP Cloud PlatformEurope (Rot) [cf-eu1] – Service Advisory Since approximately 13:53 until 14:07 UTC A general disruption is impacting the availability of applications and services

7/13/19 VMware Skyline Service – Advisor myvmware Login Availability Issue Few users will not be able to access the service or may experience trouble while accessing the service through 

Start Time: July 13, 2019, 04:40 UTC 
End Time: July 13, 2019, 08:00 UTC 

7/12/19 SAP Cloud Platform Europe (Rot) [cf-eu1] – Service Advisory Since approximately 10:32 until 13:27 UTC A general disruption is impacting the availability of applications and services

7/11/19

Availability Issues with Log Assist for VMware Skyline Log Assist functionality is impacted

Start Time: July 11, 2019 12:45 UTC
End Time: July 11, 2019 14:07 UTC

-SAP Cloud Platform Europe (Rot) [cf-eu1] – Service Advisory Since approximately 14:40 until 15:43 UTC Lifecycle management and backup operations for backing services cannot be executed 

SAP Cloud Platform US East (Sterling) [neo-us3] – Service Advisory Since approximately 19:42 until 20:15 UTC Lifecycle management operations for Java applications cannot be executed 

SAP Cloud Platform Japan (Tokyo) [neo-jp1] – Service Advisory Since approximately 20:03 until 23:30 UTC ifecycle management operations for Java applications cannot be executed 

7/10/19 VMware Cloud PKS Availability issue  User may not be able to access the service or may experience trouble while accessing the service.

Start Time: July 10, 2019 20:10 UTC 
End Time: July 10, 2019 21:10 UTC 

7/9/19

-SAP Cloud Platform Europe (Rot) – Trial [neo-eu1-trial] – Service Advisory Since approximately 15:59 UTC until 16:48 UTC, Connectivity and connections established with Connectivity are unavailable

-SAP Cloud Platform Europe (Rot) – Trial [neo-eu1-trial] – Service Advisory Since approximately 10:18 UTC until 10:36 UTC, Lifecycle management operations for Java applications cannot be executed

7/8/19

-SAP Cloud Platform Europe (Rot) – Trial [neo-eu1-trial] – Service Advisory Since approximately 06:45 UTC until 15:08 UTC, Lifecycle management operations for Java applications cannot be executed

-Google Cloud Functions Incident #19004 The Cloud Functions service is experiencing an elevated error rate with new and reconfigured functions. This includes cloud function deploys from the Firebase CLI. Incident began at 2019-07-08 13:23 and ended at 2019-07-08 20:43 (US/Pacific) lasting 7 hours 20 minutes

Google Cloud Networking Incident #19016 Cloud Networking issues in us-east1. Incident began at 2019-07-02 10:25 and ended at 2019-07-03 12:00 (US/Pacific) lasting 25 hours 34 minutes

7/3/19

-SAP Cloud Platform Europe (Frankfurt) [cf-eu10] – Service Advisory Since approximately 17:21 UTC until 18:20 UTC, Applications protected by Authorization & Trust Management (XSUAA) are not accessible

-SAP Cloud Platform Europe (Rot) – Trial [neo-eu1-trial] – Service Advisory Since approximately 14:45 UTC until 15:33 UTC, Lifecycle management operations for Java applications cannot be executed

-SAP Cloud Platform Brazil (São Paulo) [neo-br1] – Service Advisory Since approximately 07:21 UTC until 07:59 UTC, A general disruption is impacting the availability of applications and services7/2/19 Google Cloud Networking Incident #19015 Capacity loss in us-east1 region. Incident began at 2019-07-02 07:36 and ended at 2019-07-02 09:12 (US/Pacific) lasting 1 hour 35 minutes