Outages everywhere!
Fall is here, and it seems as though it's not just the leaves, but the communication and collaboration services that are dropping left and right lately. One of the most significant outages to hit the Comms environment comes from Cisco, a leader in cloud and on-premises communication solutions.
At the time of writing (28th of September), some Cisco WebEx users are still facing issues with their connections, days after the service started to experience problems on the 25th of September. The WebEx outage has been so significant, that company CEO Chuck Robbins jumped onto his Twitter account to apologize for the inconvenience, saying:
"The @WebEx outage today is unacceptable."
Chuck reassured users on the 25th that Cisco engineers were already rushing to restore their WebEx solutions and ensure that an outage of such magnitude wouldn't happen again.
The Scope of the Cisco WebEx Outage
Often, when cloud-based communication services go down, they're back up and running again within the same day. The flexible nature of the cloud makes it easier for engineers to find issues and roll out solutions rapidly. Unfortunately, Cisco seems to be having some serious trouble getting their services back online.
Cisco first began identifying problems on the 24th of September at 9:44 P.M ET. The status website for Cisco stated that WebEx Teams are "currently impacted by an ongoing service outage." Early Tuesday morning, the complaints started to roll in from customers who still couldn't access their Teams, Meeting, Calling, Care and Context solutions from Cisco.
Now, three days on, Cisco has officially reported that WebEx Meetings has been restored to "fully operational" status. However, no details are available yet as to why the system shut down in the first place. All we know is that engineers have fixed a problematic behaviour in the WebEx infrastructure.
Additionally, core messaging capabilities are now back up-and-running, with access to full messaging history for users. However, you can only access your message history if you have previously stored messages on your device. Additional message history options may not be available from the Cisco cloud services at this stage, and you won't be able to flag any messages, see user presence, or access the "one button to push" or "ability to bind a space" feature.
Rebuilding WebEx one Feature at a Time
It seems that Cisco has been forced to resort to rolling fixes out one at a time, rather than being able to simply flip a switch and get WebEx back on track. On the 27th of September, Cisco announced that "complete service restoration for messaging is an ongoing effort."
At the time of writing, WebEx Calling device activation is now back up and running, and Cisco has also reported that WebEx Desktop calling isn't affected by the incident. Hybrid call services became available on the 27th of September at 22:48 UTC. An hour later, Cisco also announced that the Care Assistant Bot for Cisco WebEx solutions is now operational. Other fixes rolled out up to now include:
27th of September:
