The rise of COVID-19 has prompted an unprecedented selection of changes in the digital marketplace. Almost overnight, the world transformed into a more physically distant one, with offices and buildings shutting down rapidly to limit the spread of the virus.
This shift drove millions of new users to collaborative tools like Microsoft Teams, Slack, Zoom Video Communications, and Google Meet. Many companies struggled to handle the rapid influx of demand. Zoom faced significant issues with security and privacy as it attempted to scale up too fast. Elsewhere, other brands fought hard to reduce potential outages.
Even Google, one of the biggest tech companies in the world, had it’s problems. However, as the Google team revealed in a recent blog post – they figured out how to thrive in this new world.
Responding to Sudden Demand
According to Google, the SRE team for Google Meet began getting requests for additional capacity mid-way through February. The company immediately began searching for additional resources, but they knew they were going to have to think further ahead too.
The SRE team began formulating a response to a need for video and meeting technology greater than anyone had ever seen. At the same time, the Google team was also in the process of transitioning into a new period of from-home working due to Covid-19. Google quickly realised that the scope of the mission was huge, and the response they issued needed to be long-term.
Eventually, the company decided to set up a series of workstreams, focusing on:
- Capacity: Finding and issuing resources
- Dependancies: Handling Meet’s infrastructure and scale
- Bottlenecks: Identifying and removing scaling limits
- Control knobs: Building new mitigations into the system
- Production changes: Delivering new capacity and servers with optimised tuning
As incident responders, Google constantly evaluated the operational structure it was using and whether it made sense. According to the team – it was a marathon, not a sprint.
Developing New Opportunities
Google rapidly jumped into action, responding to the need for new regional availability with more than 20 data centers around the world. The company quickly made use of the raw resources already available to them, which was enough to double the capacity of Meet.
"The continental peak session count dictated how much serving capacity we needed to have available" - Samantha Schaevitz, Staff Site Reliability Engineer[/caption]
