Introducing AI-Track

Today, we’re sharing our latest effort to simplify AI inference. AI Track is a universal inference and orchestration framework designed to make working with AI models easier. Its goal is to provide one unified interface that abstracts away the inference provider — whether the model runs locally or in the cloud, whether the hardware uses an M-series chip or CUDA, everything is exposed through an OpenAI-compatible interface.

The library automatically selects the most performant inference system for the given model and hardware. This includes MLX, vLLM, and Transformer-based inference libraries. At the same time, the routing system can also route requests to cloud providers. From the beginning, AI Track supports OpenRouter, Gemini, OpenAI, and Mistral.

The library abstracts AI inference into managed atoms: the AI_Model. These are managed in a shared hub instance, which also resolves model abilities and default models to return a simple OpenAI-compatible synchronous or asynchronous client. It can also manage remote providers and automatically select between them based on definable rules.

View on GitHubView on PyPI