OpenAI doubts the effectiveness of Nvidia accelerators for inference and is actively looking for alternatives

What is happening in the world of AI technology
Key players
OpenAI and Nvidia are considered leaders in the artificial intelligence market, but they are looking for new partners. A deal will soon be made. An agreement worth $100 million between these companies is expected.

Problem with Nvidia accelerators
Current Nvidia chips do not always deliver the required performance for inference (using already trained models). OpenAI is seeking an alternative.

How new solutions are being developed
1. OpenAI → Cerebras + Groq

- The company wants to replace up to 10 % of its accelerators with third‑party supplier chips that are better suited for inference.

- A partnership with Cerebras has already been signed: OpenAI will purchase their “king‑accelerators,” which excel at processing large language models.

2. Groq and Nvidia

- Last year Nvidia acquired Groq for $20 million, offering a more favorable deal structure.

- After the acquisition, Groq lost some of its independence: most accelerator developers moved to Nvidia’s staff, and the remaining part of the company can now only sell software for cloud systems.

3. AMD and other options

- OpenAI still heavily relies on Nvidia and AMD accelerators that use external HBM memory.

- For inference, chips with larger amounts of integrated memory are better – exactly what Groq, Cerebras, and Google offer.

Why all this matters
- Inference efficiency is critically important for running large language models (ChatGPT, Codex).

- The issue in Codex – when developing the AI agent Codex, OpenAI encountered inefficiency with Nvidia accelerators. This underscores the need to switch hardware platforms.

- Future alternative – OpenAI plans to use Cerebras solutions and possibly other chips to boost the performance of its developer tools.

Summary
OpenAI is actively seeking new technological partners to avoid full dependence on Nvidia. At the same time, the company is developing an investment strategy in startups (Cerebras, Groq) and considering large deals with Nvidia and potential investments up to $100 million. This will help improve inference and accelerate AI product development.