24/7/365 Sales & Support 1.866.257.8455 Login

Latest Articles

System Status: Operational

10/03/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Normal Firewall Clusters All firewall clusters operating normally
Normal Internet Connectivity Internet connectivity is operating normally

Notes: Extreme packet loss observed in Boston. The issue appears to be with packet flow betweeen the firewall cluster and internal network. STRM shows a single customer VM heavily syn flooding. Shut down network interface and traffic normalizing. I have asked JTAC to explain how this is possible, since we have syn flood screens on this interface.

System Status: Operational with Exceptions

10/03/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Impaired Firewall Clusters Connectivity issues through Boston firewall cluster
Normal Internet Connectivity Internet connectivity is operating normally

Notes: Extreme packet loss observed in Boston. The issue appears to be with packet flow betweeen the firewall cluster and internal network. JTAC is looking through logs and equipment status.

System Status: Operational with Exceptions

10/03/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Impaired Firewall Clusters Connectivity issues through Boston firewall cluster
Normal Internet Connectivity Internet connectivity is operating normally

Notes: Extreme packet loss observed in Boston. The issue appears to be with packet flow betweeen the firewall cluster and internal network. The packet loss returned after approximately an hour. Performaing a simultaneous reboot and opening a case with JTAC to get to root cause as quickly as possible.

System Status: Operational with Exceptions

10/03/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Impaired Firewall Clusters Connectivity issues through Boston firewall cluster
Normal Internet Connectivity Internet connectivity is operating normally

Notes: Extreme packet loss observed in Boston. The issue appears to be with packet flow betweeen the firewall cluster and internal network. Performed cluster failover. The issue appearst to have normalized. We are monitoring.

System Status: Operational with Exceptions

10/03/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Impaired Firewall Clusters Connectivity issues through Boston firewall cluster
Normal Internet Connectivity Internet connectivity is operating normally

Notes: Extreme packet loss observed in Boston. Looking into the cause.

System Status: Operational

09/20/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Normal Firewall Clusters All firewalls operating normally
Normal Internet Connectivity Internet connectivity is operating normally

Notes: Internap has restored their peering with Cogent.

System Status: Operational

09/19/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Normal Firewall Clusters All firewalls operating normally
Normal Internet Connectivity Internet connectivity is operating normally

Notes: We have reactivated our peering with Cogent. Internap has not, though it appears that the connection is back to normal. We will be observing closely.

System Status: Operational with Exceptions

09/19/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Normal Firewall Clusters All firewalls operating normally
Impaired Internet Connectivity Internet connectivity is impaired. Cogent peering administratively down.

Notes: While Cogent reports that the incident is over, we are waiting until after business hours to restore our peering. Internap is doing the same.

System Status: Operational with Exceptions

09/19/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Normal Firewall Clusters All firewalls operating normally
Impaired Internet Connectivity Internet connectivity is impaired. Cogent peering administratively down.

Notes: Cogent reports the following:
Customers Affected: Some Los Angeles Customers

Date: September 19th 2016
Incident Start Time: 11:35 AM PDT
Duration: 4 Hours
Associated Ticket: HD6655601

Dear Cogent Customer:

This letter is in response to the Cogent Communications network event that occurred at 11:35 AM PDT on September 19th 2016 lasting around 4 hours affecting Los Angeles area Customers.

Between 11:30 AM PDT and 3:30 PM PDT on September 19th, 2016 customers in our Los Angeles market experienced severe packet loss and latency. The issue was due to a fiber cable cut experienced by our longhaul fiber vendor in close proximity to our main Hub site in Los Angeles. This was due to unscheduled work being performed by the vendor. The Cogent NOC stayed in contact with our fiber provider, along with having our local Field Engineers on site until the fiber cut was resolved. Our vendor resolved the issue by splicing new fiber for the in-building risers.

We apologize for any impact you experienced. Please call us at 1-877-726-4368, option 2 if you have any additional questions concerning this event. Our NOC ticket number on this incident was HD6655601.

System Status: Operational with Exceptions

09/19/2016   |   0

Status System Notes
Normal vCenter vCenter is operating normally
Normal vCloud Director vCloud Director is operating normally.
Normal Virtual Machines All VMs are operating normally
Normal VMware Hosts All hosts are operating normally
Normal VMware Clusters All clusters operating normally
Normal SAN All SAN components operating normally
Normal L3 Switches All switches operating normally
Normal Firewall Clusters All firewalls operating normally
Impaired Internet Connectivity Internet connectivity is impaired. Cogent peering administratively down.

Notes: After discussions with Internap and observing customer traceroutes, we have determined the cause of the outage is Cogent. Infinitely Virtual and Internap have deactivate our peerings with Cogent. We will need to observe if this will resolve the issue.