SourceReddit r/LocalLLaMA•Stalecollected in 2h
RX 9070 ROCm Beats Vulkan with Flash Attention
#rdna4#rocm-benchmarks#flash-attnrocm-7.2.1-on-rx-9070rx-9070rocm-7.2.1flashattentionllama.cpp
💡Flash attn turns RX 9070 ROCm into PP beast—5.5x speedup revealed
⚡ 30-Second TL;DR
What Changed
ROCm + flash attn: 3,980 t/s PP512 on Qwen3-8B (5.5x over default)
Why It Matters
Unlocks RDNA4 potential for local LLM inference via proper ROCm flags, closing gap with Vulkan. Early benchmarks guide AMD GPU optimization for practitioners.
What To Do Next
Build llama.cpp with -DGGML_CUDA_FORCE_MMQ=ON -DGGML_HIP_GRAPHS=ON and run --flash-attn on ROCm 7.2.1.
Who should care:Developers & AI Engineers
Key Points
- •ROCm + flash attn: 3,980 t/s PP512 on Qwen3-8B (5.5x over default)
- •Qwen3.5-14B-A3B: ROCm 3,731 t/s PP512 (+12% vs Vulkan), Vulkan 113 t/s TG
- •Advantage shrinks at 8K context; parity there
- •Build flags: -DGGML_HIP=ON -DGGML_CUDA_FORCE_MMQ=ON -DGGML_HIP_GRAPHS=ON --flash-attn
📰
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
The weekly digest
One email a week. Unsubscribe anytime.