Inference Server

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Turn any computer or edge device into a command center for your computer vision projects.

Open-source inference server and production cluster for all the models your agent needs.

The simplest way to serve AI/ML models in production

Auto-tuned launcher for GGUF models on llama.cpp / ik_llama.cpp — OpenAI-compatible server with multi-GPU tensor-split, MoE expert placement, measured flag tuni…

ONNX Runtime Server: The ONNX Runtime Server is a server that provides TCP and HTTP/HTTPS REST APIs for ONNX inference.