🤓 Just 1 week into 2025: These 7 AI-picked stocks are up 9%+ eachUnlock Stocks

Inside Meta's scramble to catch up on AI

Published 04/25/2023, 06:13 AM
Updated 04/25/2023, 06:11 PM
© Reuters. FILE PHOTO: A security guard stands watch by the Meta sign outside the headquarters of Facebook parent company Meta Platforms Inc in Mountain View, California, U.S. November 9, 2022.  REUTERS/Peter DaSilva/File Photo

© Reuters. FILE PHOTO: A security guard stands watch by the Meta sign outside the headquarters of Facebook parent company Meta Platforms Inc in Mountain View, California, U.S. November 9, 2022. REUTERS/Peter DaSilva/File Photo

MSFT
-1.32%
GOOGL
-0.98%
NVDA
-3.00%
META
0.84%
GOOG
-1.14%

By Katie Paul, Krystal Hu, Stephen Nellis and Anna Tong

(Reuters) - As the summer of 2022 came to a close, Meta CEO Mark Zuckerberg gathered his top lieutenants for a five-hour dissection of the company's computing capacity, focused on its ability to do cutting-edge artificial intelligence work, according to a company memo dated Sept. 20 reviewed by Reuters.

They had a thorny problem: despite high-profile investments in AI research, the social media giant had been slow to adopt expensive AI-friendly hardware and software systems for its main business, hobbling its ability to keep pace with innovation at scale even as it increasingly relied on AI to support its growth, according to the memo, company statements and interviews with 12 people familiar with the changes, who spoke on condition of anonymity to discuss internal company matters.

"We have a significant gap in our tooling, workflows and processes when it comes to developing for AI. We need to invest heavily here," said the memo, written by new head of infrastructure Santosh Janardhan, which was posted on Meta's internal message board in September and is being reported now for the first time.

Supporting AI work would require Meta to "fundamentally shift our physical infrastructure design, our software systems, and our approach to providing a stable platform," it added.

For more than a year, Meta has been engaged in a massive project to whip its AI infrastructure into shape. While the company has publicly acknowledged "playing a little bit of catch-up" on AI hardware trends, details of the overhaul - including capacity crunches, leadership changes and a scrapped AI chip project - have not been reported previously.

3rd party Ad. Not an offer or recommendation by Investing.com. See disclosure here or remove ads.

Asked about the memo and the restructuring, Meta spokesperson Jon Carvill said the company "has a proven track record in creating and deploying state-of-the-art infrastructure at scale combined with deep expertise in AI research and engineering."

"We're confident in our ability to continue expanding our infrastructure's capabilities to meet our near-term and long-term needs as we bring new AI-powered experiences to our family of apps and consumer products," said Carvill. He declined to comment on whether Meta abandoned its AI chip.

Janardhan and other executives did not grant requests for interviews made via the company.

The overhaul spiked Meta's capital expenditures by about $4 billion a quarter, according to company disclosures - nearly double its spend as of 2021 - and led it to pause or cancel previously planned data center builds in four locations.

Graphic: Meta's capex boost - https://www.reuters.com/graphics/METAPLATFORMS-ARTIFICIALINTELLIGENCE/dwvkdldgkpm/chart.png

Those investments have coincided with a period of severe financial squeeze for Meta, which has been laying off employees since November at a scale not seen since the dotcom bust.

Meanwhile, Microsoft-backed OpenAI's ChatGPT surged to become the fastest-growing consumer application in history after its Nov. 30 debut, triggering an arms race among tech giants to release products using so-called generative AI, which, beyond recognizing patterns in data like other AI, creates human-like written and visual content in response to prompts.

Generative AI gobbles up reams of computing power, amplifying the urgency of Meta's capacity scramble, said five of the sources.

FALLING BEHIND

A key source of the trouble, those five sources said, can be traced back to Meta's belated embrace of the graphics processing unit, or GPU, for AI work.

3rd party Ad. Not an offer or recommendation by Investing.com. See disclosure here or remove ads.

GPU chips are uniquely well-suited to artificial intelligence processing because they can perform large numbers of tasks simultaneously, reducing the time needed to churn through billions of pieces of data.

However, GPUs are also more expensive than other chips, with chipmaker Nvidia (NASDAQ:NVDA) Corp controlling 80% of the market and maintaining a commanding lead on accompanying software, the sources said.

Nvidia did not respond to a request for comment for this story.

Instead, until last year, Meta largely ran AI workloads using the company's fleet of commodity central processing units (CPUs), the workhorse chip of the computing world, which has filled data centers for decades but performs AI work poorly.

According to two of those sources, the company also started using its own custom chip it had designed in-house for inference, an AI process in which algorithms trained on huge amounts of data make judgments and generate responses to prompts.

By 2021, that two-pronged approach proved slower and less efficient than one built around GPUs, which were also more flexible in running different types of models than Meta's chip, the two people said.

Meta declined comment on its AI chip's performance.

As Zuckerberg pivoted the company toward the metaverse - a set of digital worlds enabled by augmented and virtual reality - its capacity crunch was slowing its ability to deploy AI to respond to threats, like the rise of social media rival TikTok and Apple-led ad privacy changes, said four of the sources.

The stumbles caught the attention of former Meta board member Peter Thiel, who resigned in early 2022, without explanation.

3rd party Ad. Not an offer or recommendation by Investing.com. See disclosure here or remove ads.

At a board meeting before he left, Thiel told Zuckerberg and his executives they were complacent about Meta's core social media business while focusing too much on the metaverse, which he said left the company vulnerable to the challenge from TikTok, according to two sources familiar with the exchange.

Meta declined to comment on the conversation.

CATCH-UP

After pulling the plug on a large-scale rollout of Meta's own custom inference chip, which was planned for 2022, executives instead reversed course and placed orders that year for billions of dollars worth of Nvidia GPUs, one source said.

Meta declined to comment on the order.

By then, Meta was already several steps behind peers like Google (NASDAQ:GOOGL), which had begun deploying its own custom-built version of GPUs, called the TPU, in 2015.

Executives also that spring set about reorganizing Meta's AI units, naming two new heads of engineering in the process, including Janardhan, the author of the September memo.

More than a dozen executives left Meta during the months-long upheaval, according to their LinkedIn profiles and a source familiar with the departures, a near-wholesale change of AI infrastructure leadership.

Meta next started retooling its data centers to accommodate the incoming GPUs, which draw more power and produce more heat than CPUs, and which must be clustered closely together with specialized networking between them.

Graphic: Meta's U.S. data center status - https://www.reuters.com/graphics/METAPLATFORMS-ARTIFICIALINTELLIGENCE/mopakdkmzpa/chart.png

The facilities needed 24 to 32 times the networking capacity and new liquid cooling systems to manage the clusters' heat, requiring them to be "entirely redesigned," according to Janardhan's memo and four sources familiar with the project, details of which have not previously been disclosed.

3rd party Ad. Not an offer or recommendation by Investing.com. See disclosure here or remove ads.

As the work got underway, Meta made internal plans to start developing a new and more ambitious in-house chip, which, like a GPU, would be capable of both training AI models and performing inference. The project, which has not been reported previously, is set to finish around 2025, two sources said.

Carvill, the Meta spokesperson, said data center construction that was paused while transitioning to the new designs would resume later this year. He declined to comment on the chip project.

TRADE-OFFS

While scaling up its GPU capacity, Meta, for now, has had little to show as competitors like Microsoft (NASDAQ:MSFT) and Google promote public launches of commercial generative AI products.

Chief Financial Officer Susan Li acknowledged in February that Meta was not devoting much of its current compute to generative work, saying "basically all of our AI capacity is going towards ads, feeds and Reels," its TikTok-like short video format that is popular with younger users.

According to four of the sources, Meta did not prioritize building generative AI products until after the launch of ChatGPT in November. Even though its research lab FAIR, or Facebook (NASDAQ:META) AI Research, has been publishing prototypes of the technology since late 2021, the company was not focused on converting its well-regarded research into products, they said.

As investor interest soars, that is changing. Zuckerberg announced a new top-level generative AI team in February that he said would "turbocharge" the company's work in the area.

Chief Technology Officer Andrew Bosworth likewise said this month that generative AI was the area where he and Zuckerberg were spending the most time, forecasting Meta would release a product this year.

3rd party Ad. Not an offer or recommendation by Investing.com. See disclosure here or remove ads.

Two people familiar with the new team said its work was in the early stages and focused on building a foundation model, a core program that later can be fine tuned and adapted for different products.

Carvill, the Meta spokesperson, said the company has been building generative AI products on different teams for more than a year. He confirmed that the work has accelerated in the months since ChatGPT's arrival.

MSFT: A Bull or Bear Market Play?

Don't miss out on the next big opportunity! Stay ahead of the curve with ProPicks – 6 model portfolios fueled by AI stock picks with a stellar performance this year..

In 2024 alone, ProPicks' AI identified 2 stocks that surged over 150%, 4 additional stocks that leaped over 30%, and 3 more that climbed over 25%. That's an impressive track record.

With portfolios tailored for Dow stocks, S&P stocks, Tech Stocks, and Mid Cap stocks, you can explore various wealth-building strategies.

So if MSFT is on your watchlist, it could be very wise to know whether or not it made the ProPicks lists.

Unlock ProPicks now

Latest comments

Derick LimApr 25, 2023, 14:30
Metaccan just continue announcing laying off employees plan to push up the shares without a single advertisement profit ......and the IBs analysts are happily upgrading Meta based on predictions.......
Loading next article…
Risk Disclosure: Trading in financial instruments and/or cryptocurrencies involves high risks including the risk of losing some, or all, of your investment amount, and may not be suitable for all investors. Prices of cryptocurrencies are extremely volatile and may be affected by external factors such as financial, regulatory or political events. Trading on margin increases the financial risks.
Before deciding to trade in financial instrument or cryptocurrencies you should be fully informed of the risks and costs associated with trading the financial markets, carefully consider your investment objectives, level of experience, and risk appetite, and seek professional advice where needed.
Fusion Media would like to remind you that the data contained in this website is not necessarily real-time nor accurate. The data and prices on the website are not necessarily provided by any market or exchange, but may be provided by market makers, and so prices may not be accurate and may differ from the actual price at any given market, meaning prices are indicative and not appropriate for trading purposes. Fusion Media and any provider of the data contained in this website will not accept liability for any loss or damage as a result of your trading, or your reliance on the information contained within this website.
It is prohibited to use, store, reproduce, display, modify, transmit or distribute the data contained in this website without the explicit prior written permission of Fusion Media and/or the data provider. All intellectual property rights are reserved by the providers and/or the exchange providing the data contained in this website.
Fusion Media may be compensated by the advertisers that appear on the website, based on your interaction with the advertisements or advertisers.
© 2007-2025 - Fusion Media Limited. All Rights Reserved.