Forbes

The New Frontier Of LLM Inference: Where The Next Tenfold Gains Will Come From

Shakti P. Singh, Principal Engineer at Intuit and former OCI model inference lead, specializing in scalable AI systems and LLM inference. Generative models are rapidly making inroads into enterprise ...
The Manila Times

MOREH Demonstrates Production-Ready LLM Inference on Tenstorrent Galaxy, Achieving DGX A100-Class Performance with Improved Cost Efficiency

SANTA CLARA, Calif., May 2, 2026 /PRNewswire/ -- Moreh, an AI infrastructure software company, led by CEO Gangwon Jo, ...
Yahoo Finance

Red Hat Launches the llm-d Community, Powering Distributed Gen AI Inference at Scale

Forged in collaboration with founding contributors CoreWeave, Google Cloud, IBM Research and NVIDIA and joined by industry leaders AMD, Cisco, Hugging Face, Intel, Lambda and Mistral AI and university ...
SiliconANGLE

Red Hat sees inference as AI’s next battleground — with Kubernetes at the core

As AI demands drive orders-of-magnitude increases in token consumption, the need for scalable, production-grade Kubernetes inference has never been greater. “What we realized is that AI is being ...
Le Lézard

Cacheon Launching Open Inference Arena for LLM Serving Optimization

Cacheon today announced its open inference competition platform, with mainnet deployment planned later this month. The platform creates an open arena where developers and researchers compete to build ...
SDxCentral

AI inference crisis: Google engineers on why network latency and memory trump compute

Google researchers have warned that large language model (LLM) inference is hitting a wall amid fundamental problems with memory and networking problems, not compute. In a paper authored by ...
VentureBeat

AI inference costs dropped up to 10x on Nvidia's Blackwell — but hardware is only half the equation

Lowering the cost of inference is typically a combination of hardware and software. A new analysis released Thursday by Nvidia details how four leading inference providers are reporting 4x to 10x ...

Lumai Launches the World’s First Optical Computing System for Real-Time, Billion-Parameter LLM Inference

Lumai, the optical compute company addressing scalable AI, today announced its Lumai Iris inference server – the world’s first optical computing system to successfully run billion-parameter large ...