The Supercomputing Power Behind The Power of OpenAI

Yet another development: this week, OpenAI announced the arrival of its ChatGPT-4 model.

With the relentless super buzz around ChatGPT and the gold rush for the implementation of AI, there are two main concerns for tech experts: One, will it lead to a robot revolution? Two, how much processing power or storage will it need?

For the UC&C space facing its ever-integrating — and in the positive sense, decreasing — circles of the platform, apps, call centre and additional services and products, the questions arise of how such pre-trained functionality can be implemented or developed.

Some deployment questions are already being answered, with Teams Premium adding Intelligent Recap, Live Translation and Meeting Templates to a platform already integrated with AI. Features such as transcript provision, action point-making and chapter-making to improve efficiency and productivity will become standard, as we discussed in our article on what ChatGPt means for the platform. We also learnt how developers could build apps for the platform’s app gallery, ready to plug a chatbot into various aspects of communication and collaboration.

• What is ChatGPT? The Latest Chatbot Superstar
• Top Vendors Using AI in Video Technology
• Microsoft Integrates ChatGPT Technology into Bing

Are the Robots Taking Over?

These kind of opportunities for app-making have now been given more impetus by OpenAI, which provides access to an API for developers who want to play with the Whisper transcription language model and offers a ten times cheaper ChatGPT model for enterprises.

But does this mean a ‘robot revolution’? The latest thinking is that AI is still an assistant tool, and it is beholden to the information it is allowed to seek and how it has been requested. Meanwhile, Microsoft often uses the term ‘co-pilot’.

Satya Nadella, CEO and Chairman of Microsoft, described a developer testing ChatGPT to write 85% of his code: “He (the developer) is still the pilot. He just has a co-pilot.”

The people hired for their AI expertise will understand it’s about working with and optimising the models rather than training the models to replace them. Tasks undertaken, such as menial or automated ones to improve production flow, or researching and writing, are intended to save time and assist. Like search, mastering ChatGPT is about the quality of the inputting, after all.

Inventor and Co-founder of Apple, Steve Wozniak, declared in a recent interview for CNBC’s Squawk Box that the human brain and AI work differently and that regarding artificial intelligence he would agree with the first word ‘artificial’ but not with the ‘intelligence’ part of the term. He stated:

“A computer can sit down and play a billion games of chess and remember everything. No human could do that.”

Wozniak said he couldn’t imagine machines being programmed to have emotions such as love or solve life problems but only provide smart solutions to problems offered; he said: “It is following algorithms.”

So let’s run with that for now. Next issue:

What About the Power behind AI Power?

Chatbots need vast amounts of computing energy. Regarding ChatGPT, OpenAI CEO and Co-Founder Sam Altman famously tweeted last December: “We will have to monetise it at some point; the compute costs are eye-watering.”

More massive investment announcements from Microsoft followed in the first part of this year as ChatGPT’s user tally soared to exceed $100 million. The main aim of the investment: is supercomputing at scale and developing and deploying specialised systems. This, in turn, is due to speed up OpenAI’s research in AI technology, build out Azure’s AI infrastructure and thus help customers build and deploy AI applications globally. Ducks in a row: win, win, win.

Microsoft AI in Development

For decades, Microsoft had already developed AI models for languages, from automatic spell-checking in Word to AI tools that translate captions in PowerPoint and Microsoft Translator in over 100 languages. As these AI capabilities improved, the tech giant leveraged its expertise in high-performance computing to extend its infrastructure in the Azure cloud, enabling customers to use AI tools to build, train, and deploy custom AI applications.

Nidhi Chappell, Microsoft Head of Product for Azure High-Performance Computing and AI, remarked: “One thing we learned from research is that the larger the model, the more data you have and the longer you can train, the better the accuracy of the model is.”

“So, there was a strong push to get bigger models trained for a longer period, which means not only do you need to have the biggest infrastructure, you have to be able to run it reliably for a long period.”

OpenAI’s Big Idea

So when OpenAI approached Microsoft with their audacious pitch needing a processing superpower, Microsoft AI researchers used more powerful graphics processing units (GPUs) for more complex workloads. Over time, the researchers realised the size of the supercomputing infrastructure that OpenAI would require: the industrialisation of processing.

According to Phil Waymouth, Senior Director, Strategic Partnerships, Microsoft’s Office of the CTO, the gearshift proved a catalyst; he said:

“That shift from large-scale research happening in labs to the industrialisation of AI allowed us to get the results we’re starting to see today,”

For startup OpenAI, the industrial supercomputing capabilities were a springboard for the ChatGPT project, as Greg Brockman, President and Co-Founder of OpenAI, explained: “Because these jobs span thousands of GPUs, you need to ensure you have reliable infrastructure, and then you need to have the network in the backend to communicate faster and be able to do that for weeks on end.”

Brockman confirmed:

“Co-designing supercomputers with Azure has been crucial for scaling our demanding AI training needs, making our research and alignment work on systems like ChatGPT possible.”

The Infrastructure for Inferencing

When Microsoft and OpenAI worked together, they built an infrastructure which included thousands of NVIDIA AI-optimized GPUs interconnected for high-performance computing. The network was high-throughput and low-latency, based on NVIDIA Quantum InfiniBand communications. 

Brockman added: “This is not something that you just buy a whole bunch of GPUs, hook them together, and they’ll start working together. There is a lot of system-level optimisation to get the best performance, which comes with a lot of experience over many generations.”

When a chatbot finishes a task, say, writes an email or a proposal, it is called an inference. Delivering these AI capabilities to customers worldwide needs an AI infrastructure optimised for millions of inferences. Microsoft has deployed GPUs for inference across its Azure data centres in over 60 regions. So it’s not just the immediate chatbot tech we see on the screen that is the momentous shift; it’s the industrial supercomputing behind it, the power behind the power. 

As Eric Boyd, Microsoft Corporate Vice-President for AI Platform, pointed out: “Now, when other people come to us and want the same style of infrastructure, we can give it to them because that’s the standard way we do it.”

This AI-optimised infrastructure is now cemented in the Azure cloud, including virtual machines, connected compute, and storage optimised for AI workloads. The tech giant is in a good position, as Scott Guthrie, Executive Vice President of the Cloud and AI Group at Microsoft, testified; he said: “Only Microsoft Azure provides the GPUs, the InfiniBand networking and the unique AI infrastructure necessary to build these types of transformational AI models at scale, which is why OpenAI chose to partner with Microsoft.”

“Azure is the place now to develop and run large transformational AI workloads.”