LMSYS Org Launches SGLang Consortium with NVIDIA, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk to Advance Open-Source AI Inference

LMSYS Org, the nonprofit behind the open-source AI engine SGLang, today launched the SGLang Consortium with founding members NVIDIA, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk, who will ...

Hedra Launches Model Factory, an Autonomous Post-Training and Inference Platform for Real-World AI

Hedra, an a16z-backed research company building models and infrastructure for multimodal and physical AI, today launched ...
MarkTechPost

DeepSeek Open-Sources DeepInfer, the Inference Framework Behind DeepSeek-V4.1-Flash

DeepSeek open-sourced DeepInfer, its production inference framework for DeepSeek-V4.1-Flash, reporting over 25,000 decode tokens per second.
Crypto Briefing

DeepSeek open-sources DeepInfer, the inference engine behind its production models

DeepSeek open-sourced DeepInfer, its Apache-2.0 inference framework for V4.1-Flash, with support for NVIDIA SM100 and Huawei Ascend 950 chips.
New Atlas

Next-level AI engine comes top in LLM speed showdown

Responses to AI chat prompts not snappy enough? California-based generative AI company Groq has a super quick solution in its LPU Inference Engine, which has recently outperformed all contenders in ...

Is your local LLM slow? The 'inference engine' changes the speed

Previously, I mentioned that if you have a machine with 64GB of memory or more, I recommend Strata over Ollama.So, what ...
Yahoo Finance

DigitalOcean Launches Inference Engine with New Capabilities for Production AI, Including Inference Router for Efficient Scaling of Agentic Workloads

The above button links to Coinbase. Yahoo Finance is not a broker-dealer or investment adviser and does not offer securities or cryptocurrencies for sale or facilitate trading. Coinbase pays us for ...
Hosted on MSN

This dev made a llama with three inference engines

Developers looking to gain a better understanding of machine learning inference on local hardware can fire up a new llama engine.… Software developer Leonardo Russo has released llama3pure, which ...
HackerNoon

TokenPulse: Building the Leadership Team Behind Cost-Efficient AI Inference

Discover how TokenPulse combines GPU infrastructure, inference optimization, and global partnerships to deliver ...
The Next Web

DeepSeek open-sources DeepInfer, the engine that runs its AI models

DeepSeek open-sources DeepInfer, its production inference engine. It runs on Nvidia and Huawei chips and supports V4.1-Flash for now.
MarkTechPost

Liquid AI Launches RIPL, a Real-Time Inference Engine for Sub-50 ms LFMs

Liquid AI describes RIPL as optimizing the full path, from request arrival to completed response. In several workloads, Liquid AI plots the same model, LFM2.5-350M, on vLLM and on RIPL. RIPL’s p95 ...
VentureBeat

OpenInfer raises $8M for AI inference at the edge

OpenInfer has raised $8 million in funding to redefine AI inference for edge applications. It's the brain child of Behnam Bastani and Reza Nourai, who spent nearly a decade of building and scaling AI ...