Welcome Inkling by Thinking Machines

Summary

Inkling, released by Thinking Machines, is a decoder-only multimodal Mixture-of-Experts model with 975 billion total parameters and 41 billion active parameters. It accepts native image, audio, and text inputs and supports a 1-million-token context window. The model was trained on 45 trillion tokens spanning text, images, audio, and video, with an emphasis on cross-modal reasoning and subsequent domain adaptation through fine-tuning.

Its architecture incorporates several distinctive choices. Positional information is supplied through relative attention rather than rotary embeddings: each layer learns a per-token, per-head relative feature that is combined with distance information before the attention computation. Decoder layers alternate between global attention and sliding-window attention in a 5:1 ratio, with the final layer always using global attention. A short 1-D convolution (SConv) operates over the hidden states to capture local patterns, while the MoE router activates six routed experts plus two always-on shared experts. Vision and audio inputs are handled by lightweight hierarchical MLP towers that embed image patches and discretized mel-spectrogram bins directly into the token stream.

The release supplies production-grade deployment artifacts. The BF16 checkpoint requires roughly 2 TB of VRAM; the calibrated NVFP4 variant reduces this to about 600 GB. Day-zero support is available in Transformers (≥5.14), SGLang, vLLM, and llama.cpp, together with tensor-parallel SLURM scripts for multi-node clusters. Speculative decoding is enabled by additional MTP layers that act as drafters, and community 1-bit GGUF quantizations further lower memory footprint. Inference endpoints can be started with a single command in each framework, exposing OpenAI-compatible APIs.

Reasoning effort is controlled at the tokenizer level with discrete settings ranging from “none” to “max.” Post-training recipes cover supervised fine-tuning, distillation via logit matching, and reinforcement learning with the ECHO algorithm inside the tinker environment. Vibe evaluations on expert-level reasoning tasks and audio benchmarks illustrate token usage and accuracy across effort levels, providing practical guidance for production use.

Why it matters

Directly addresses ML Engineers with concrete architecture details, latency/memory trade-offs, distributed serving patterns, and fine-tuning workflows for a frontier multimodal model, enabling immediate experimentation and production deployment.