Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation
06:00 · July 3, 2026 · arXiv cs.AI RSS
Developing high-performance kernels for Neural Processing Units (NPUs) is a critical industry bottleneck, requiring developers to manually navigate implicit hardware constraints and strict memory hierarchies. While large language models offer immense automation potential, they fail catastrophically on NPUs due to a fundamental lack of hardware-specific priors. Naively transplanting code snippets from similar NPU kernels may pass the compiler, but it consistently triggers runtime crashes and performance degradation by blindly violating underlying hardware constraints. To overcome this, we introduce Hawk, a training-free framework that harnesses hardware-aware knowledge through three core modules: (1) Run-Time Knowledge Synthesis Module, which employs a Triple-Part Executable Knowledge Representation to inherently couple the error context with executable semantics; (2) Bottleneck-Aware Knowledge Retrieval Module, which implements a 2D-Retrieval paradigm to project queries into orthogonal syntactic and hardware-aligned semantic spaces; and (3) Effect-Driven Knowledge Distillation Module, which leverages LLM-driven semantic arbitration to continuously distill the knowledge by pruning errors and consolidating redundancies based on the empirical execution feedback. Extensive evaluations on real-world NPU workloads demonstrate that Hawk elevates generation accuracy from 49.4% to 80.0%, while achieving up to a 2.2x execution speedup over state-of-the-art baselines.
Summary
Developing high-performance kernels for Neural Processing Units remains a persistent bottleneck because developers must manually respect implicit hardware constraints and a strict multi-tier memory hierarchy. Large language models, despite their code-generation capabilities, produce kernels that frequently violate these constraints. Even when code compiles, it often triggers runtime crashes or runs nearly twenty times slower than equivalent GPU implementations under comparable theoretical capacity. Empirical checks show that over 75 percent of such failures trace to low-level hardware misuses such as invalid API calls or mismatched memory layouts.
Hawk addresses this gap with a training-free framework that maintains a dynamic knowledge base of existing NPU kernels and retrieves hardware-aware guidance at generation time. Three coordinated modules supply the missing priors. The Run-Time Knowledge Synthesis Module converts newly observed execution outcomes into a three-part representation that pairs indexing triggers, natural-language rationales for hardware limits, and verified syntactic templates. The Bottleneck-Aware Knowledge Retrieval Module projects each query into two orthogonal spaces—one syntactic for API compliance and one hardware-aligned for memory and parallelism constraints—thereby locating root-cause strategies rather than superficially similar code. The Effect-Driven Knowledge Distillation Module then uses compilation and runtime feedback to prune erroneous entries and remove redundant patterns, keeping the context window focused on empirically validated constraints.
Evaluations on real-world Ascend NPU workloads show that these mechanisms raise generation accuracy from 49.4 percent to 80.0 percent while delivering up to 2.2 times faster execution than prior state-of-the-art baselines. By avoiding both model retraining and manual intermediate-representation engineering, Hawk offers a scalable route to automated yet hardware-compliant NPU kernel development.
Why it matters
This research is highly relevant for Dutch AI hardware and infrastructure researchers, particularly those working within the Netherlands' strong semiconductor and edge computing sectors. It provides an actionable, advanced methodology for optimizing NPU performance, aligning with EU goals for efficient AI deployment.
