We got local models to triage the OpenClaw repo for FREE!*

Summary

A semi-agentic workflow lets local open-weight models classify incoming GitHub issues and pull requests for the OpenClaw repository. The system receives each item’s title, body and a truncated diff, then decides whether to inspect the codebase through a restricted read-only shell before emitting structured labels such as local_models, inference_api or tool_calling. The shell, called reposhell, exposes only commands such as ls, cat and grep on the target repository, blocking any write or network action that a prompt-injected issue might otherwise trigger.

Two models were evaluated on production hardware: gemma-4-26b-a4b and qwen3.6-35b-a3b. Both run under vLLM with NVFP4 quantization, prefix caching, an FP8 KV cache and the CUTLASS MoE backend on an NVIDIA GB10 with 128 GB unified memory. On a 330-item evaluation set whose labels were adjudicated from multiple GPT-5.5 and Opus 4.8 runs, Gemma delivered higher recall and lower wall-clock time per item while Qwen produced higher precision and fewer false positives. Throughput reached several hundred output tokens per second at concurrency 16, completing the full set in roughly 7.5 minutes.

The surrounding pipeline keeps inference only for the classification step. Once labels are produced, deterministic rules route notifications to Discord, avoiding unnecessary model calls for routine routing. The same harness is also used in a parallel OpenClaw cron job that periodically compares local labels against a GPT-5.5 reference, surfacing false positives and negatives for ongoing monitoring. The approach demonstrates that medium-sized local models can perform zero-shot, high-throughput triage tasks with usable accuracy, provided tool access is tightly constrained and inference is reserved for steps that genuinely require it.

Why it matters

It provides a practical, hands-on guide to deploying local models for agentic tasks, addressing critical production concerns like inference optimization, secure tool execution, and cost-efficiency. This aligns well with the EU's focus on data sovereignty and local AI deployment.