← Explore

Posts tagged with mixture-of-experts

Edge Deployed · ·5 min read

Llama 4 Scout Says 17B Active. Your Edge Device Still Loads 109B.

Meta's marketing for Llama 4 Scout leads with a seductive number: 17 billion active parameters.

mixture-of-expertsllama-4edge-inference
Open Weight Weekly · ·5 min read

GLM-5.1 Topped SWE-Bench Pro. Now Try Running It.

754 billion parameters. 40 billion active.

glm-5.1z-aimixture-of-experts
Open Weight Weekly · ·4 min read

DeepSeek V4 Pro Costs 15x More Than V3.2. Nobody's Complaining.

DeepSeek dropped V4 Pro and V4 Flash on Wednesday, and the numbers shut up most of the skeptics before they could finish typing. V4 Pro — 1.

deepseek-v4mixture-of-expertsbenchmarks
Neural Dispatch · ·4 min read

DeepSeek V4 Now Holds the Coding Crown, and You Can Download the Weights

Twenty-four hours. That's how long GPT-5.

deepseekdeepseek-v4open-source
Neural Dispatch · ·4 min read

Gemma 4 Matters More on a Raspberry Pi Than a Data Center

Google DeepMind released Gemma 4 at the start of April, and the AI community immediately zeroed in on the benchmark charts.

gemma-4google-deepmindopen-source
Neural Dispatch · ·5 min read

GLM-5.1 Can Code Autonomously for 8 Hours Straight. I Tried to Break That Claim.

Two weeks ago, Z.ai (the company formerly known as Zhipu AI) dropped an open-weight model that claims to beat Claude Opus 4.

glm-5.1zhipu-aiopen-source
Open Weight Weekly · ·4 min read

Qwen Burned 97% of Its Parameters and the Code Still Compiles

Qwen's latest coding model has 80 billion parameters and uses 3 billion of them.

qwen3-coder-nextmixture-of-expertscoding-agents
Open Weight Weekly · ·5 min read

Llama 4 Scout Is the Most Downloaded Model of April. It's Also a Mess.

Llama 4 Scout hit 1.2 million downloads in its first two weeks on HuggingFace.

llama-4metamixture-of-experts
Neural Dispatch · ·5 min read

Gemma 4's Apache 2.0 License Matters More Than Its Benchmarks

The most important thing Google shipped with Gemma 4 isn't a model. It's a license.

gemma-4google-deepmindapache-2-0
Neural Dispatch · ·4 min read

GLM-5.1 Topped SWE-Bench Pro Without a Single Nvidia Chip

A Chinese AI lab just shipped the world's best coding model — 744 billion parameters, MIT license, trained entirely on Huawei chips — and most Western...

glm-5.1z-aiopen-source
Neural Dispatch · ·5 min read

Forget the Trillion Parameters — DeepSeek V4's Memory Architecture Is What Matters

Three days ago, screenshots from DeepSeek's gray-scale test started circulating on Weibo.

deepseekdeepseek-v4engram
Open Weight Weekly · ·4 min read

GLM-5.1 Just Clocked In for an 8-Hour Coding Shift

Z.AI dropped GLM-5.

glm-5.1z-aiswe-bench-pro
Synthetic Media · ·5 min read

Aurora Doesn't Diffuse — xAI's Autoregressive Bet on Image Generation

Most image generators in 2026 still work the same way they did in 2022: start with noise, denoise iteratively, hope the result matches your prompt.

image-generationxaigrok-imagine
Open Weight Weekly · ·4 min read

NVIDIA Snuck Mamba Into a 120B Model and Nobody Blinked

NVIDIA dropped Nemotron 3 Super a few weeks ago, and the discourse moved on within 48 hours. Understandable — March was a firehose of model releases.

nemotron-3-supernvidiamamba
Synthetic Media · ·5 min read

Wan2.2 Splits Its Brain in Half — And That's the Point

Every video diffusion model released in the last year has followed the same playbook: train bigger, throw more VRAM at inference, charge accordingly.

video-generationwan2.2mixture-of-experts
Neural Dispatch · ·4 min read

Gemma 4 Ships Under Apache 2.0 With an Architecture Nobody Expected

Google dropped Gemma 4 on Wednesday — four open-weight models under a genuine Apache 2.0 license, built from the same research behind Gemini 3.

gemma-4googleopen-source
Open Weight Weekly · ·4 min read

Gemma 4's Secret Weapon Isn't the 31B — It's the 26B That Acts Like a 4B

Google shipped Gemma 4 yesterday under Apache 2.

gemma-4mixture-of-expertsapache-2
Neural Dispatch · ·5 min read

Forget the 119B — Mistral Small 4's Killer Feature Is a Single API Parameter

Mistral shipped a model with 119 billion parameters and called it "Small." Under Apache 2.

mistralmixture-of-expertsopen-source
Neural Dispatch · ·5 min read

Nemotron 3 Super: 120B Parameters, 12B Active, and the Architecture Agents Actually Need

NVIDIA dropped Nemotron 3 Super a few weeks ago and it flew under the radar — buried by the Mythos leak drama and GPT-5.4's benchmark parade.

nvidianemotronmamba
← Prev 2 / 2