๐ฆReddit r/LocalLLaMAโขStalecollected in 63m
Dual GPUs Hack for 16GB VRAM Users
๐กCheap VRAM hack runs 30B models at 19t/s on mixed GPUs
โก 30-Second TL;DR
What Changed
Combine 16GB + 6GB GPUs via Vulkan devices
Why It Matters
Config tweaks like split-mode=layer and no-mmap minimize RAM use.
What To Do Next
Run `llama-server --list-devices` to identify Vulkan IDs and test dual-GPU config.
Who should care:Developers & AI Engineers
Key Points
- โขCombine 16GB + 6GB GPUs via Vulkan devices
- โขConfig: n-gpu-layers=999, split-mode=layer, np=1
- โข22GB total VRAM runs 27B Q4_K_M model
- โข19.21 t/s eval, 186 t/s prompt at 128k context
๐ฐ
Weekly AI Recap
Read this week's curated digest of top AI events โ
๐Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA โ