Major Cloud Providers – Monthly Outage Recap February

MS Azure

2/27/19 USGov Virginia – Service Availability Between 07:38 and 09:50 EST on 27 Feb 2019, a subset of customers may have experienced degraded performance or timeouts while accessing Azure resources.

2/22/19 Virtual Machines (Classic)Between 04:40 UTC and 10:50 UTC on 22 Feb 2019 a subset of customers using Virtual Machines (Classic) may have experienced failures/ high latency when attempting to complete service management operations. Some customers may also have experienced downstream impact to API Management and Backup.

2/20/19 RCA – SQL Services – West EuropeBetween 09:40 UTC and 17:15 UTC on 20 Feb 2019 a subset of customers using SQL Services (inclusive of Azure DB for MariaDB, MySQL, and PostgreSQL, SQL DB, and SQL Data Warehouse) in West Europe experienced issues performing service management operations and/or experienced service availability issues following scaling operations. Symptoms may have included but were not limited to:
• Service management operations returning failure notifications
• Server and database create, drop & scale operations may result in “deployment failed” errors
• Failures when creating databases through SQL Script
• Databases may become unavailable after performing scaling operations.

Note: This issue was impacting all types of SQL service deployments (e.g. Elastic Pool, Single and Managed Instances).

2/18/19 RCA – Erroneous Browser Warning – Azure Germany PortalBetween 14:10 and 17:20 UTC on 18 Feb 2019, customers attempting to login to the Azure Germany portal may have received erroneous warnings stating “Deceptive site ahead.”

2/11/19 RCA – Azure Kubernetes Service – East USBetween 15:00 and 22:05 UTC on 11 Feb 2019, a subset of customers using Azure Kubernetes Service (AKS) in East US may have experienced the following symptoms:

  • Intermittent failures with cluster service management operations – such as create, update, delete, and scaling. 
  • Issues performing workload management operations.
  • Brief cluster downtime.

2/8/19 RCA – Azure IoT HubBetween 08:16 and 14:12 UTC on 08 Feb 2019, a subset of customers using Azure IoT Hub may have received failure notifications or high latency when performing service management operations via the Azure Portal or other programmatic methods.2/1/19 Active Directory – MitigatedBetween 08:05 and 10:00 UTC on 01st Feb 2019, a small subset of users in certain countries in Europe including France, Netherlands, Hungary, Czech Republic may have experienced intermittent issues while accessing functionality in Azure Portal, Azure Active Directory B2C, Azure Active Directory Privileged Identity Management, Managed Service Identity, Azure RBAC and Microsoft Teams.

AWS

2/27/19

Dow Jones risk screening watchlist exposed on misconfigured AWS serverA list maintained by Dow Jones, owned by News Corp. since 2007, was found by security researcher Bob Diachenko and announced Wednesday. He found an exposed Dow Jones database on an AWS Elasticsearch instance.

Amazon CloudFrontBetween 10:09 AM and 1:23 PM PST customers may have experienced longer than usual propagation times while making changes to CloudFront configurations. 

2/19/19 VMware Cloud on AWS Emergency Maintenance impacting provisioning of new SDDCsImpact: Provisioning new VMC SDDCs may fail. Start time : Feb 19, 19.35 PM UTC End time : Feb 19, 20.05 PM UTC

2/6/19 VMware Cloud Services: Console Login intermittent availabilityVMware Cloud on AWS Service is experiencing Availability issue. Impact: User may not be able to access the service or may experience trouble while accessing the service.  Start Time: February 06, 2019 12:35 PM UTC End Time: February 06, 2019 01:41 PM UTC

Other Platforms

2/26/19

-Google Cloud Functions Incident #19001 We’ve received a report of an issue with Google Cloud Function deployments seeing increased errors. Incident began at 2019-02-26 13:00 and ended at 2019-02-26 15:21 (US/Pacific).

-Google Cloud Datastore Incident #19002 Seeing increased write error rates with Google Cloud Datastore in europe-west1 due to timeouts. Began at 2019-02-26 03:02 and ended at 2019-02-26 08:32 (US/Pacific).

-Google Kubernetes Engine Incident #19004 Users affected by this issue may receive HTTP 5XX errors when listing clusters with the GetClusters API Endpoint. Users will also be unable to resize, upgrade or repair their clusters in the affected regions. Incident began at 2019-02-26 15:30 and ended at 2019-02-26 20:53 (US/Pacific).

-SAP Cloud Platform China (Shanghai) [neo-cn1] – Service Advisory Since approximately 12:14 UTC on 26 Feb 2019, customers may have experienced a disruption on the SAP Cloud Platform. Impact:  Applications and services are unavailable. End time 12:19 UTC on 26 Feb 2019. 

VMware Skyline Service Availability issue experiencing Degraded performance issue. Impact: This is only impacting users that authenticate through my vmware. Start Time: February 26, 2019 04:15 PM UTC End Time: February 26, 2019 06:05 PM UTC

VMware Cloud Services: Console Login intermittent availability Impact: Customers might experience intermittent login failures. UPDATE: The users will be able to login to VMware Cloud Services Console however the dependent back-end like Support, Billing Services Will be impacted. Start Time: February 26, 2019 05:00 PM UTC End Time: February 26, 2018 07:25 PM UTC

2/21/19

-IBM Cloud Unplanned maintenance on EU-Cloud – EU Cloud broker Services will be unavailable SERVICES/COMPONENTS AFFECTED:

Cloudant NoSQL DB IMPACT: Due to an issue with the 5.0.4 broker upgrade on eu-cloud, Cloudant will require a maximum of 5 mins interruption to EU-cloud service to complete this deploy. START TIME February 21, 2019 3:27 AM PST END TIME February 21, 2019 3:34 AM PST

-SAP Cloud Platform Australia (Sydney) [neo-ap1] – Service Advisory Since approximately 15:46 UTC on 21 Feb 2019, customers may have experienced a disruption on the SAP Cloud Platform. Impact:  Applications and services may be unavailable. End time 16:25 UTC on 21 Feb 2019. 

2/19/19 SAP Cloud Platform Europe (Rot) [neo-eu1] – Service Advisory Since approximately 15:30 UTC on 19 Feb 2019, customers may have experienced a disruption on the SAP Cloud Platform. Impact: Portal may be unavailable. End time 17:51 UTC

2/16/19 VMware Kubernetes Engine: Cluster Create issues Impact: User may not be able to create Clusters. Start Time: February 16, 2019 02:49 AM UTC End Time: February 16, 2019 04:07 AM UTC

2/15/19 SAP Cloud Platform US Central (IA) [cf-us30] – Service Advisory Since approximately 00:17 UTC on 15 Feb 2019, customers may have experienced a disruption on the SAP Cloud Platform. Impact: Provisioning of backing services may be unavailable. End time 06:18 UTC

2/13/19

-Google Cloud Networking Incident #19003 Instances in us-central1-b, us-central1-c or us-central1-f may have seen increased packet loss between other regions and to the internet from 07:37 to 07:46 US/Pacific.

-Google Kubernetes Engine Incident #19003 GKE may clear certain add-ons from the configuration UI after a cluster upgrade. Incident began at 2019-02-13 17:06 and ended at 2019-02-13 17:13 (US/Pacific).

VMware Kubernetes Engine: Cluster Create issues Impact: Users may not be able to create clusters. Start Time: February 13, 2019 03:45 AM UTC  End Time: February 13, 2019 05:10 AM UTC

2/9/19 VMware Cloud backend service Intermittent Availability issue Impact: User may not be able to access or experienc trouble when logging into the the VMware Cloud Services. Start Time: February 09, 2019 6:30 AM UTC End Time: February 06, 2019 07:25 AM UTC

2/8/19 IBM Cloud Services experiencing read out time SERVICES/COMPONENTS AFFECTED:

– Cloud Foundry Application Management

– Cloud Foundry Applications

 IMPACT:

– Users might have experienced read out time

– Access to running Cloud Foundry based applications

– Application routing traffic

REGION: London. START TIME
February 8, 2019 3:55 AM PST END TIME
February 8, 2019 4:03 AM PST

2/7/19 IBM Cloud Authentication Failures SERVICES/COMPONENTS AFFECTED: Bluemix Cloudfoundry IMPACT: Authentication Failures REGION: Dallas. START TIME February 7, 2019 6:30 AM PST END TIME February 7, 2019 7:12 AM PST

2/6/19 IBM Cloud Customers may not be able to access resources that depend on Resource Group policies REGION: Dallas.  

SERVICES/COMPONENTS AFFECTED: Identity and Access Management IMPACT: Customers may receive a ‘Deny Access” or 403 message when requesting access to a resource in a Resource Group. START TIME February 6, 2019 1:15 PM PST END TIME February 6, 2019 2:27 PM PST

2/5/19

VMware Cloud backend service Availability issue Impact: User may not be able to access or experienc trouble when logging into the the VMware Cloud Services. Start Time: February 05, 2019 11:50 AM UTC End Time: February 05, 2019 12:20 PM UTC

VMware Cloud backend service Availability issue VMware Cloud backend services were experiencing Availability issue with console login. Start Time: February 05, 2019 01:01 PM UTC End Time: February 05, 2019 01:05 PM UTC

-Google Cloud Storage Incident #19001 Google Cloud Storage experienced elevated error rates averaging 10% across the European multi-region for GET, PUT, and DELETE requests. Began at 2019-02-05 09:21 and ended at 2019-02-05 10:17 (US/Pacific).

2/4/19

VMware Cloud backend service Availability issue Impact: User may experience trouble when logging into the the VMware Cloud Services. Start Time: February 04, 2019 11:15 AM UTC End Time: February 04, 2019 12:05 PM UTC

VMware Issues with Subscription,Usage and Billing Service APIs Please be advised that we are experiencing an issue with VMware Cloud Services. Impact: Customers will not able to view the Subscription, Billing, Support and Usage related details. Start Time: February 04, 2019 07:19 AM UTC End Time: February 04, 2019 07:30 AM UTC

-IBM Cloud Public_London: Time Out issues with service provisioning and binding calls SERVICES/COMPONENTS AFFECTED: BSS Provisioning Broker. START TIME February 4, 2019 5:32 PM PST END TIME February 4, 2019 7:12 PM PST

2/3/19 SAP Cloud Platform Europe (Rot) [neo-eu1] – Service Advisory Since approximately 14:00 UTC on 03 Feb 2019, customers may have experienced a disruption on the SAP Cloud Platform. End time 16:00 UTC.

Impact: 
– Creation of new virtual machines might not be possible.
– Lifecycle management operations for Java applications may not be possible 

2/2/19 IBM Cloud Users may experience issues authenticating to the platform SERVICES/COMPONENT AFFECTED: IBM ID Authentication. REGIONS: Dallas, London. START TIME February 2, 2019 9:34 PM PST END TIME February 2, 2019 10:05 PM PST