THE ESSENTIALS IN 30 SECONDS

  • A Microsoft Foundry post dated November 5 shows Foundry Local serving Qwen3.8-27B through ONNX Runtime, connected to GitHub Copilot via its Bring Your Own Key option.
  • With 4-bit weights the model takes about 18 GB, so it runs on a single NVIDIA GPU with 24 GB of memory, with up to 64K tokens of context, or up to 262K on 32 GB cards.
  • The 73.0 Terminal Bench 2.1 and 61.7 SWE-bench Pro scores cited are Alibaba's own August evaluations, not independent results.

Microsoft's Foundry team has shown how to run Alibaba's Qwen3.8-27B coding model entirely on a developer's own computer and plug it into GitHub Copilot. In a post dated November 5, authors Baiju Meswani, Emma Ning and Tianlei Wu describe Foundry Local, Microsoft's on-device inference runtime, serving the model through ONNX Runtime, according to coverage by AI Weekly and daily.dev. The pitch is that prompts and source code stay on the machine and no per-token API fees apply.

How it works

Copilot connects through its Bring Your Own Key feature, which lets users point it at a model provider of their choice. Developers install the Foundry Local command-line tool (version 0.13.0 or later), load the model, read the local API endpoint, and then pick Foundry Local and Qwen3.8-27B in Copilot's model selector. CUDA and cuDNN runtimes have to be installed separately. Complete AI Training reports that the current release targets NVIDIA GPUs on Windows, with installation through WinGet.

Quantizing the weights to 4 bits brings the model to roughly 18 GB, so the minimum is a single NVIDIA GPU with 24 GB of memory. Context reaches 64K tokens on such cards and up to the model's 262K maximum on GPUs with 32 GB or more. The Foundry team lists the optimizations behind this: DFlash speculative decoding, quantization of the KV cache, prefix caching for concurrent requests, CUDA graph fusions and CPU offloading. Microsoft says it tested on an RTX 5090 and a DGX Spark and found Foundry Local generated faster than llama.cpp at every prompt length, with the gap widening beyond 131K tokens; the coverage we reviewed did not reproduce the throughput figures.

What the scores mean

The headline numbers, 73.0 on Terminal Bench 2.1 and 61.7 on SWE-bench Pro, are the ones Alibaba published when it released Qwen3.8-27B under the Apache 2.0 license in mid-August. DataNorth notes they are vendor evaluations, run in a Claude Code harness, and that Alibaba modified some SWE-bench Pro tasks, so they are not directly comparable with other leaderboards. They also describe the full model, not necessarily the 4-bit build running locally.

Coverage notes that a 27-billion-parameter local model reasons less well than the strongest cloud models, and Microsoft frames the setup as part of a hybrid workflow. Mac and AMD GPU support, more local coding models and deeper Copilot integration are on the roadmap, with no dates given. We could not open Microsoft's original post directly; details here are as reported by the outlets listed.