OpenAI unveils custom chip it designed with Broadcom to boost its AI infrastructure

Taiwan Semiconductor Manufacturing Co

Yesterday 14:00

OpenAI plans to deploy Jalapeño by the end of this year
Broadcom's CEO said Jalapeño matches Nvidia Blackwell and Google's tensor processing units
OpenAI designed Jalapeño for inference to answer chatbot queries
By Max A. Cherney

    SAN FRANCISCO, June 24 (Reuters) - OpenAI showed off the company's first custom artificial intelligence chip designed in conjunction with Broadcom AVGO.O on Wednesday, as it seeks to speed its development of its infrastructure. 
AI labs such as OpenAI and Anthropic are struggling to obtain enough computing horsepower to run the latest, most powerful chatbots and coding apps. Some, such as OpenAI, have turned to developing in-house chips in order to reduce the cost and create an alternative to Nvidia's NVDA.O graphics processing units (GPUs) that are commonly used for AI.
OpenAI's engineers designed the chip, called Jalapeño, together with Broadcom to perform a specific AI task known as inference, during which data is crunched in order to answer a user's query to a chatbot like ChatGPT.
The chip made by the team is as good as the Blackwell chips made by Nvidia or the tensor processing units designed by Alphabet's GOOGL.O Google, Broadcom CEO Hock Tan said in an interview with Reuters.
The Jalapeño processor is designed to work speedily and efficiently with the large language models (LLMs) that power many AI applications, OpenAI hardware chief Richard Ho said.
"It will be performant on, we think, all kind of future iterations of LLMs," Ho told Reuters. 
The company plans to deploy Jalapeño by the end of this year, and it is the first step in a multi-generation chip development plan, OpenAI said.
Canadian electronics manufacturer Celestica CLS.TO will build the server systems, which, like the chips will be used only by OpenAI. 
The San Francisco company said it has samples of the chip running in its labs and they were operating at the target power and performance with the company's GPT-5.3-Codex-Spark AI model. 
 It took the company's engineers roughly nine months to complete the chip design before it sent it to Taiwan's TSMC 2330.TW for manufacturing, in part because of using AI to speed specific aspects of the process, OpenAI said. 
Reuters first reported OpenAI was exploring making its own chip in 2023. 
To make their own in-house chips, Meta Platforms META.O, Amazon AMZN.O and Google have turned to the likes of Broadcom and Marvell MRVL.O, which provide specific design services and intellectual property that can be difficult to replicate in-house. 
     Anthropic is weighing building an AI chip of its own, sources told Reuters in April.
At the moment, however, because of the AI-related surge in memory demand, Broadcom's profit margin on the custom chips is not as high as some of the other chips it makes, such as networking switches, Tan said. 
AI chips require large amounts of high-bandwidth memory, which challenges Broadcom's margins on custom AI chip products, Tan said. Tan said South Korea's SK Hynix 000660.KS and Samsung Electronics 005930.KS supply Broadcom with memory chips. 

 (Reporting by Max A. Cherney in San Francisco; Editing by Jamie Freed)
 ((max.cherney@thomsonreuters.com; 415-404-2697; @chernandburn on Twitter/X))