Tag: #amd-gpu12 results
Zyphra released ZAYA1-8B, an open 8B-parameter mixture-of-experts reasoning model trained on AMD Instinct MI300 GPUs, achieving competitive benchmarks against GPT-5-High and DeepSeek-V3.2. Available under Apache 2.0 on Hugging Face for free download and customization. It features innovative MoE++ architecture for superior efficiency.
kernel-anvil profiles GGUF model shapes on AMD GPUs and generates optimal llama.cpp kernel configs for 2x decode speedup without recompilation. It targets RDNA3 GPUs like 7900 XTX, achieving 27 tok/s on Qwen3.5-27B Q4_K_M. Usage involves quick profiling and a small llama.cpp patch.
ZINC is a new LLM inference engine written in Zig, enabling 35B models on $550 AMD GPUs via Vulkan. It loads GGUF models, achieves 7.1 tok/s on RDNA4, and addresses AMD consumer GPU gaps. Repo at github.com/zolotukhin/zinc.
A user built a PyTorch-based tiled attention mechanism as a flash-attention alternative for unsupported AMD MI50 GPUs (gfx906), enabling video generation without OOM. Inspired by llama.cpp, it uses query chunking, softmax fallbacks, and optimizations like BF16-to-FP16 conversion. Pure PyTorch, no custom kernels needed.
Benchmark shows dual AMD RX 7900 XTX GPUs achieving 123 tok/s token generation and up to 3,829 tok/s prompt processing on Qwen3.5-35B-A3B Q4_K_M using llama.cpp Vulkan backend. It outperforms single 7900 XTX and rivals high-end NVIDIA cards. vLLM on ROCm underperforms significantly with bugs.
Hugging Face blog announces MachinaCheck, a multi-agent system designed for CNC manufacturability checks. The project leverages AMD MI300X accelerators to enable advanced AI-driven manufacturing analysis. It showcases practical applications of multi-agent AI in industrial workflows.
Hugging Face blog showcases fine-tuning MedQA, a clinical AI model for medical QA, on AMD's ROCm platform. This eliminates the need for NVIDIA's CUDA, enabling training on AMD GPUs. It highlights open-source alternatives for AI development.
Hipfire is a new inference engine tailored for all AMD GPUs, featuring mq4 quantization. The creator shares quantized models on Hugging Face. Benchmarks on Localmaxxing reveal major speedups for RDNA3 and older cards.
Tutorial enables running Qwen3.5-397B-A13B with vLLM on 8x AMD R9700 GPUs using MXFP4 quantization. Achieves 30 t/s single request, up to 100 t/s batched. Includes Dockerfile patches and launch script for high throughput.
Latest AMD GPU firmware and llama.cpp build yield major Vulkan performance gains on Strix Halo. Tested Qwen3.5-35B-A3B-Q8_0 on Debian Linux with ROCm 7.12. Vastly improved prompt processing over prior benchmarks.