Together GPU Clusters 新增自動擴展與自癒

🔑 增強重點摘要

• •Together AI is reportedly pursuing a $1 billion funding round amid these infrastructure upgrades.[1]
• •Autoscaling is powered by Kubernetes Cluster Autoscaler, which monitors GPU-constrained workloads and automatically provisions or decommissions nodes based on real-time demand.[1][2]
• •Self-healing involves a three-click process where the control plane cords, drains, and recreates failed nodes, with automatic acceptance tests ensuring clusters pass before marking as ready.[2]
• •GPU Clusters support NVIDIA H100, H200, B200, and GB200 GPUs with InfiniBand networking, scaling from 8 to over 4,000 GPUs for distributed training and inference.[3]

🛠️ 技術深入

• •Autoscaling uses Kubernetes Cluster Autoscaler for dynamic node provisioning/deprovisioning based on GPU demand, targeting variable inference and bursty training jobs.[1][2]
• •Observability features a dedicated Grafana instance with dashboards for DCGM GPU metrics, InfiniBand/NIC networking telemetry, storage I/O, and Kubernetes health.[2]
• •Self-healing node repair: Cordon and drain failed nodes, recreate on new/existing hosts, run acceptance tests (detailed in documentation) before readiness.[2]
• •Orchestration options include managed Kubernetes (Kubeadm-based, HA control plane, node autoscaling, Cert Manager) and Slurm on Kubernetes for precise gang scheduling and SSH access.[3]
• •Performance enhanced by Together Kernel Collection (including ThunderKittens), achieving 90% faster training on B200 (15,264 tokens/s/GPU) vs H100 (8,080 tokens/s/GPU) for 70B Llama model.[3]

🔮 前景展望AI analysis grounded in cited sources

⏳ 時間線