scrobble.life
#ai

Qwen 3.8 27B Released - My observations

Marky@themarkymark (81)10d

image.png

Qwen released their 3.8 version of their most popular 27B model. This model is extremely popular because it can be run on consumer hardware and it is a dense model, so all 27 billion parameters activate for every response. Most models these days are a Mixture of Experts (MOE) which typically only 1/10th of the parameters are active at any time.

The main purpose of MOE models is to reduce the computing demand to run them. You still need the same amount of memory to run a dense or MOE model, but you can get a lot more performance from a MOE model.

As 27B is a dense model, it punches well above it's size with it performing as good as 300B model in some cases. This makes it extremely popular for local AI users.

Qwen 3.6 27B was very popular for this reason, and Qwen never released 3.7 weights making many believe that was the last model we would be able to run ourselves. Today that changes with the 3.8 27B weights dropping on Huggingface this morning.

I ran Qwen 3.6 27B vs Qwen 3.8 27B side by side in my llmcompare benchmark. This benchark tests both models with real world problems and uses ChatGPT 5.5 to judge the results.

In my testing I found Qwen 3.8 27B to be considerably slower and use a lot more tokens to arrive at an answer, but the answers consistently were a lot better than Qwen 3.6 27B. In fact, this is one of the more impressive intelligence lifts I've seen.

Nine out of 10 results went to Qwen 3.8 27B but the answers were significantly better than Qwen 3.6 27B.

Here you can see the full summary of a 10 task run.

image.png

Qwen 3.8 27B comes in at almost 3x the tokens to achieve the answer, but an average score of 8.838 vs 6.862 is pretty significant.

Ignore the token speed, as I am running Qwen 3.8 27B on dual RTX 6000 Pros and Qwen 3.6 27B is running off openrouter. This also affects the time, if I ran both models locally Qwen 3.6 27B would been significantly faster making the time gap even larger.

Here is a breakdown one one of the tests where Qwen 3.8 27B just blew the doors off Qwen 3.6 27B but took around 600% longer to arrive at the answer.

image.png

Comments · 2

  • @sircork(66)· 9d

    Whats your Qwen vs Gemma take on local hardware Marky? I find Gemma WAY faster on old CPU based compute where I run open models on an old laptop, so very anecdotal and not scientific but Gemma is actually usable on an old Lenovo in ollama, Qwen had a literal mental breakdown just trying to answer "what model variant are you?" as a first prompt. It went into a mental meltdown about how to respond that was so bizarre I actually saved it as a text file while it looped over its identity crisis about how to respond to me for over 15 minutes and I had to actually break it and stop it. Gemma responded with its model variant in about 11 seconds.

    VERY unscientific but using it is what really matters. I wanted to use Qwen but it just won't run on moldy old hardware and Gemma does ok there.

    So I am curious what your benches say about them compared to each other on accuracy?

  • @thelogicaldude(74)· 10d

    I use a combo of 3.6 35B a3b and 3.6 27B for different things so will certainly be downloading and giving it a shot.