Tag: #moe-architecture6 results
Alibaba launched Qwen3.5-397B-A17B, an open-weight MoE model with 397B total parameters but only 17B active per token, outperforming its trillion-parameter Qwen3-Max on benchmarks. It offers 19x faster decoding at 256K context, 60% lower running costs, and native multimodal capabilities from scratch training on text, images, and video. The hosted version supports up to 1M tokens.
Alibaba open-sourced three new Qwen3.5 models: Qwen3.5-35B-A3B, Qwen3.5-122B-A10B, and Qwen3.5-27B, outperforming larger predecessors and GPT-5 mini on multiple benchmarks. These MoE and dense models run on consumer GPUs, with Qwen3.5-Flash hosted on Alibaba Cloud at 0.2 yuan per million input tokens. They excel in agent tasks, coding, math, and multimodal reasoning.
Alibaba has introduced the open-source Qwen3.5 series designed for native multimodal agents. The inaugural model is a ~400B parameter vision-language model (VLM) featuring reasoning capabilities via a hybrid mixture-of-experts (MoE) and Gated Delta Networks architecture. It excels in understanding and navigating user interfaces, surpassing previous VLMs.
Indian AI lab Sarvam released two MoE LLMs built from scratch: 30B-A1B for low-latency apps and 105B-A9B for demanding tasks. Models will be open-weight on Hugging Face with API access soon. The 105B model beats Gemini 2.5 Flash on Indic benchmarks and DeepSeek R1 on many others.
One year ago, DeepSeek R1 launched as a 671B parameter MoE model. Today, the 26B Gemma 4 MoE is impressively competitive despite being 25 times smaller. The post expresses excitement for the future of local LLMs.
User critiques outdated MythoMax for slow NSFW roleplay buildup. Seeks models enabling NSFW chats in 2-3 messages for diverse scenarios. Inquires about MOE architecture, top roleplay rankings, and LLM orchestrators.