☁️AWS Machine Learning Blog•較早收集於 4m
vLLM 多 LoRA 提升 AWS 上 MoE 模型服務
💡Serve dozens of fine-tuned MoE models efficiently on AWS with vLLM's new multi-LoRA – scales production inference.
⚡ 30-Second TL;DR
有什麼變化
在 vLLM 中實現 MoE 模型的多 LoRA 推論
為什麼重要
此功能大幅提升多個微調 LLM 的推論可擴展性，在 AWS 上降低生產部署成本。AI 團隊現可無需成比例增加資源即可處理多樣模型變體。
下一步行動
Deploy vLLM on SageMaker to test multi-LoRA inference with your fine-tuned MoE models.
誰應關注：Developers & AI Engineers
關鍵要點
- •在 vLLM 中實現 MoE 模型的多 LoRA 推論
- •進行核心層級優化以提升效能
- •支援高效服務數十個微調模型
- •使用 GPT-OSS 20B 在 SageMaker 和 Bedrock 上示範
🧠 深度解析
背景與延伸：來自公開資料，非原文內容。引用 9 個來源。
🔑 增強重點摘要
- •vLLM's NFS-LoRA replicates full LoRA adapters A and B across devices, while S-LoRA shards them using tensor parallelism to reduce memory but introduces communication overhead via all-gather and all-reduce operations[1].
- •GPT-OSS 20B is a specific open-source MoE model used to demonstrate multi-LoRA serving, alongside other supported MoE models like Mixtral, Deepseek-V2, and Deepseek-V3[3][4][5].
- •Multi-LoRA in vLLM builds on S-LoRA techniques for serving thousands of adapters by replacing matrix computations with XW + XAB and optimizing for transformer-based LLMs[1].
🛠️ 技術深入
- •In NFS-LoRA (vLLM's default pre-S-LoRA), adapters A and B are fully replicated on each device, avoiding sharding but increasing memory usage[1].
- •S-LoRA shards LoRA matrices A and B across GPUs using tensor parallelism, computing XAB in parallel but requiring all-gather for inputs and all-reduce for outputs, with merged query/key/value projections reducing operations to one larger all-gather[1].
- •vLLM supports MoE models requiring an 'experts' attribute in sparse MoE blocks, compatible with architectures like TransformersForCausalLM, and enables tensor/pipeline/expert parallelism for distributed inference[3][4][6].
🔮 前景展望AI analysis grounded in cited sources
Multi-LoRA on MoE will enable serving over 100 fine-tuned models per base model on AWS by mid-2026
AWS optimizations extend vLLM's S-LoRA scaling to thousands of adapters, demonstrated with GPT-OSS 20B on SageMaker/Bedrock for dozens already.
Communication overhead in sharded LoRA will drive further kernel fusions in vLLM V1+
Evaluations show significant all-gather costs even at low ranks, prompting merged computations as in current vLLM implementations.
⏳ 時間線
2024-07
vLLM partners with Meta for Llama 3.1 support including FP8 quantization and pipeline parallelism
2024-07
vLLM hosts fifth meetup with AWS, advancing upstreaming efforts
2025-01
vLLM V1 alpha released with 1.7x speedup, prefix caching, and multimodal enhancements
2026-02
AWS implements multi-LoRA inference for MoE models in vLLM with kernel optimizations on SageMaker and Bedrock
📎 來源 (9)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
📰
AI 週報
閱讀本週精選 AI 大事摘要 →
👉相關動態
AI 策展新聞聚合。所有內容版權歸原始發布者所有。
原始來源： AWS Machine Learning Blog ↗
每週 AI 簡報
每週一封，可隨時退訂。