Search

Tag: #open-source-llm20 results

Covenant-72B: Largest Decentralized GPU Model

Covenant-72B: Largest Decentralized GPU Model

Covenant AI released Covenant-72B, the largest model trained on decentralized permissionless GPU nodes. It employs SparseLoco, built on DiLoCo, to cut communication overhead via reduced sync frequency, local AdamW optimizer, and top-K sparsification addressing bandwidth issues.

Reddit r/LocalLLaMACommunityMar 17#open-source-llm
Page 1 of 2