SetupAI
繁
SourceStalecollected in 9h

Ornith-1.0 model family released on Hugging Face

Read original on Reddit r/LocalLLaMA
#llm#moe#open-weights

New high-parameter MoE model family claims SOTA performance; worth testing for your next fine-tuning project.

30-Second TL;DR

What Changed

Ornith-1.0 released in 9B, 31B, 35B MoE, and 397B MoE variants

Why It Matters

The inclusion of a 397B MoE model provides a new high-parameter option for researchers and enterprise users. If benchmarks hold, this could be a significant contender in the open-weights ecosystem.

What To Do Next

Download the 9B or 35B MoE version to benchmark against your current production models for latency and accuracy.

Who should care:Researchers & Academics

Key Points

  • •Ornith-1.0 released in 9B, 31B, 35B MoE, and 397B MoE variants
  • •Claims state-of-the-art benchmark performance
  • •Available for download on Hugging Face

Deep Insight

AI-generated analysis for this event — not the original article.

Enhanced Key Takeaways

  • •The Ornith-1.0 architecture utilizes a novel 'Dynamic Sparse Attention' mechanism designed to reduce inference latency by 30% compared to standard dense transformers.
  • •DeepReinforce-AI has implemented a proprietary 'Reinforcement Learning from Synthetic Feedback' (RLSF) training pipeline, moving away from traditional human-annotated RLHF.
  • •The 397B MoE variant employs a unique 8x49B configuration, allowing it to fit within a 4-node H100 cluster for distributed inference.
  • •Initial community evaluations on the Hugging Face discussion boards suggest the model exhibits significantly lower hallucination rates in long-context reasoning tasks compared to Llama-3-70B.
  • •The model weights are released under a custom 'DeepReinforce Community License' which permits commercial use but requires attribution and sharing of fine-tuned derivatives.

Competitor Analysis

Architecture
Ornith-1.0 (397B MoE)
Sparse MoE
Llama 3.1 (405B)
Dense Transformer
Mixtral 8x22B
Sparse MoE
Licensing
Ornith-1.0 (397B MoE)
Custom Community
Llama 3.1 (405B)
Llama 3.1 Community
Mixtral 8x22B
Apache 2.0
Primary Strength
Ornith-1.0 (397B MoE)
Inference Efficiency
Llama 3.1 (405B)
General Reasoning
Mixtral 8x22B
Open Ecosystem

Technical Deep Dive

  • Architecture: Sparse Mixture-of-Experts (MoE) with top-2 expert routing.
  • Context Window: Native 128k token support using RoPE (Rotary Positional Embeddings) scaling.
  • Training Data: Trained on 15 trillion tokens of high-quality synthetic and curated web data.
  • Quantization: Official support for FP8 and INT4 quantization via AutoGPTQ and bitsandbytes integration.
  • Hardware Requirements: Minimum 2x A100 (80GB) for the 9B variant; 8x H100 (80GB) recommended for the 397B MoE variant.

Future ImplicationsAI analysis grounded in cited sources

DeepReinforce-AI will likely face legal scrutiny regarding their RLSF training data sources.
The reliance on synthetic feedback often involves training on outputs from other proprietary models, which may violate the terms of service of those providers.
Ornith-1.0 will trigger a shift toward RLSF-based training methodologies in open-source model development.
The reported performance gains and reduced reliance on expensive human annotation make this a highly attractive alternative for smaller research labs.

Timeline

2025-11
DeepReinforce-AI founded with a focus on automated reinforcement learning pipelines.
2026-03
Internal testing of Ornith-0.5 prototype begins.
2026-06
Ornith-1.0 model family officially released on Hugging Face.
Read original article on Reddit r/LocalLLaMA

Weekly AI Recap

Read this week's curated digest of top AI events →

PostLinkedIn

AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/LocalLLaMA ↗

This is a summary, not the original. Read the source, or get the weekly briefing.

The weekly digest

One email a week. Unsubscribe anytime.