llama.cpp Runs Qwen 3.5 122B on 4x MI50 GPUs

💡llama.cpp fork runs 122B Qwen on AMD MI50s – breakthrough for local giant models

⚡ 30-Second TL;DR

What Changed

Merged Turbo3 + gfx906 forks into fresh llama.cpp

Why It Matters

Advances AMD GPU support for ultra-large local models, key for cost-effective inference.

What To Do Next

Clone the new llama.cpp fork and test Qwen3.5 122B on MI50 GPUs.

Who should care:Developers & AI Engineers

AI-generated analysis for this event.

•The MI50 GPU, based on the Vega 20 architecture (gfx906), officially reached end-of-life status for ROCm support several years ago, making this implementation a significant community-driven effort to bypass hardware obsolescence.
•Running a 122B parameter model on 64GB of total VRAM (4x 16GB) requires heavy quantization (likely 4-bit or lower) and offloading strategies, as the model weights alone exceed the available VRAM capacity.
•The successful merge of the Turbo3 and gfx906 forks demonstrates the viability of using community-maintained 'ROCm-legacy' patches to extend the utility of older enterprise-grade AMD hardware for modern LLM inference.

Architecture: The MI50 uses the Vega 20 GPU, which lacks the matrix core hardware (WMMA) found in newer CDNA architectures, relying instead on traditional FP16/FP32 compute units.
Memory Constraints: With 16GB per card, the 4-card setup provides 64GB of VRAM. A 122B model at 4-bit quantization typically requires ~65-70GB, suggesting the implementation likely utilizes aggressive KV-cache offloading to system RAM or extreme quantization (e.g., 3-bit).
Software Stack: The 'gfx906' fork is a specialized llama.cpp branch that manually enables ROCm support for older architectures that the official upstream ROCm/llama.cpp releases have deprecated.
Communication: Performance is heavily bottlenecked by the PCIe generation and interconnect speed between the MI50 cards, as they lack the high-speed Infinity Fabric links found in newer MI-series accelerators.

Community-maintained ROCm forks will continue to be the primary driver for LLM inference on legacy AMD enterprise hardware.

Official AMD ROCm support cycles are too short to keep older enterprise hardware viable for the rapidly evolving LLM ecosystem.

Memory-constrained inference will shift toward hybrid VRAM/System RAM offloading techniques.

As model sizes grow, the cost of high-VRAM enterprise GPUs will force users to optimize for heterogeneous memory architectures.

2018-11

AMD releases the Radeon Instinct MI50, the first 7nm enterprise GPU.

2023-05

ROCm support for gfx906 (Vega 20) is officially deprecated in newer ROCm releases.

2024-02

Community developers begin maintaining independent 'gfx906' forks for llama.cpp to restore functionality.

2026-03

User successfully merges Turbo3 and gfx906 forks to run Qwen 3.5 122B on MI50 hardware.

Weekly AI Recap

Read this week's curated digest of top AI events →

Same topic

Explore #amd

Same product