SourceReddit r/LocalLLaMA•Stalecollected in 63m
Dual GPUs Hack for 16GB VRAM Users
#multi-gpu#local-llm#vr-am-hackllama-serverllama-servervulkanqwen3.6-27b
💡Cheap VRAM hack runs 30B models at 19t/s on mixed GPUs
⚡ 30-Second TL;DR
What Changed
Combine 16GB + 6GB GPUs via Vulkan devices
Why It Matters
Config tweaks like split-mode=layer and no-mmap minimize RAM use.
What To Do Next
Run `llama-server --list-devices` to identify Vulkan IDs and test dual-GPU config.
Who should care:Developers & AI Engineers
Key Points
- •Combine 16GB + 6GB GPUs via Vulkan devices
- •Config: n-gpu-layers=999, split-mode=layer, np=1
- •22GB total VRAM runs 27B Q4_K_M model
- •19.21 t/s eval, 186 t/s prompt at 128k context
📰
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
The weekly digest
One email a week. Unsubscribe anytime.