Qwen-3.8-Flash-Next is a big deal and a herald of things to come.
Flash models are models distilled down from full size models, in this case from Qwen-3.8-Max, a 2.4T model requiring some 8 datacenters GPUs to run. The Flash model instead features 125B parameters, roughly 5% of the original model of which again only 6B are active per token (4%).
Compare that to 27B active parameters of Qwen-3.8-27B dense, the current favorite for local AI and you can see the difference: More knowledge (parameters) available but fewer used for inference, dropping cost/increasing speed.
Supplementing the performance are an additional 51B ngram embeddings supplementing for lost “knowledge”in the distillation process that can be offloaded to system memory(!)
The result is a model that can run in an optimized NV4 quant with 256k context, generous KV cache and multimodal image and video capabilities (!) on 94GB of VRAM - enough for a single Blackwell RTX 6000 at about 80 t/s or 128k context supporting multiple concurrent lanes with 130 t/s -
Faster than it’s 27B brother and significantly better. Lower quants could conceivably run on something like a 3090 while the 4bit quant should still be able to push 50t/s on a 128GB+ mac studio
None of this would matter if the performance didn’t match up and, all benchmarks aside, it’s phenomenal in my real world tests:
I regularly port old DOS games to the web using AI and no local model has been able to handle the tasks involved, in particular reverse engineering, driving a debugger and writing the required complex systems code until now.
Qwen-38-27B, with a lot of handholding, was able to somewhat brute force itself through one of these tasks for me earlier this week, but it wasn’t economic, too slow and not competitive with cheap models like gpt-5.6-luna.
Qwen-38-Flash-Next is something else. It’s the first local model that reaches Opus performance for me, maybe not 4.8 but better than 5 (lol Opus 5) and better than 4.6 which for many people was the “good enough” crossover point after which improvements become mostly optional.
“Good enough for challenging engineering work” has arrived on local machines and judging from the buzz around GLMs flash model, there’s still a lot of room to go here All of which is very bad news for US labs, because whatever HBM shortages are stopping companies and consumers from having local capabilities are guaranteed to be temporary and these models scream commoditisation.
Another takeaway: I was able to squeeze this model onto a 96GB card with a handful of prompts less than 24g after release.
Gpt-5.6-sol was able to deploy it, benchmark configurations, work through bugs and make custom patches to the inference runtime (SGlang) autonomously.
This matters A LOT because it shows you that “what’s in the weights” doesn’t constrain the technology much anymore: GPT-5.6 does not know a thing about Qwen3.8, its architecture and trickery and it could still autonomously optimise it from search alone.
Image: Test Drive III (1990) port to mobile web in progress using Qwen-3.8-flash-next on a local RTX-6000 MaxQ
Qwen-3.8-Flash-Next is a big deal and a herald of things to come. Flash models are models distilled down from full size models, in this case from Qwen-3.8-Max, a 2.4T model requiring some 8… | Georg Zoeller | 11 comments
Flash models are models distilled down from full size models, in this case from Qwen-3.8-Max, a 2.4T model requiring some 8 datacenters GPUs to run. The Flash model instead features 125B parameters, roughly 5% of the original model of which again only 6B are active per token (4%). Compare that to 27B active parameters of Qwen-3.8-27B dense, the current favorite for local AI and you can see the difference: More knowledge (parameters) available but fewer used for inference, dropping cost/increasing speed. Supplementing the performance are an additional 51B ngram embeddings supplementing for lost “knowledge”in the distillation process that can be offloaded to system memory(!) The result is a model that can run in an optimized NV4 quant with 256k context, generous KV cache and multimodal image and video capabilities (!) on 94GB of VRAM - enough for a single Blackwell RTX 6000 at about 80 t/s or 128k context supporting multiple concurrent lanes with 130 t/s - Faster than it’s 27B brother and significantly better. Lower quants could conceivably run on something like a 3090 while the 4bit quant should still be able to push 50t/s on a 128GB+ mac studio None of this would matter if the performance didn’t match up and, all benchmarks aside, it’s phenomenal in my real world tests: I regularly port old DOS games to the web using AI and no local model has been able to handle the tasks involved, in particular reverse engineering, driving a debugger and writing the required complex systems code until now. Qwen-38-27B, with a lot of handholding, was able to somewhat brute force itself through one of these tasks for me earlier this week, but it wasn’t economic, too slow and not competitive with cheap models like gpt-5.6-luna. Qwen-38-Flash-Next is something else. It’s the first local model that reaches Opus performance for me, maybe not 4.8 but better than 5 (lol Opus 5) and better than 4.6 which for many people was the “good enough” crossover point after which improvements become mostly optional. “Good enough for challenging engineering work” has arrived on local machines and judging from the buzz around GLMs flash model, there’s still a lot of room to go here All of which is very bad news for US labs, because whatever HBM shortages are stopping companies and consumers from having local capabilities are guaranteed to be temporary and these models scream commoditisation. Another takeaway: I was able to squeeze this model onto a 96GB card with a handful of prompts less than 24g after release. Gpt-5.6-sol was able to deploy it, benchmark configurations, work through bugs and make custom patches to the inference runtime (SGlang) autonomously. This matters A LOT because it shows you that “what’s in the weights” doesn’t constrain the technology much anymore: GPT-5.6 does not know a thing about Qwen3.8, its architecture and trickery and it could still autonomously optimise it from search alone. Image: Test Drive III (1990) port to mobile web in progress using Qwen-3.8-flash-next on a local RTX-6000 MaxQ
linkedin.com