Tag: #generalization12 results
Zhang et al. 2016 showed standard NNs with SGD generalize on real labels but memorize random ones, defying VC dimension and Rademacher complexity. This challenged statistical learning theory's dominance for explaining deep nets. The paper marked a crisis for DL theory optimism.
Physical Intelligence, a robotics startup, has unveiled π0.7, a new model that can perform tasks it was never explicitly taught. The company describes it as an early step toward a general-purpose robot brain.
Challenges claim that SFT memorizes while RL generalizes in reasoning. Cross-domain generalization depends on optimization dynamics, data quality, and base model capability. Reveals dip-and-recovery training pattern, asymmetric effects on reasoning vs. safety.
This empirical study analyzes over 2,000 RL publications to map the evolution from physical simulations to LLM-driven agents. It proposes a multi-dimensional taxonomy revealing a bifurcation into 'Semantic Prior' (LLM-dominated) and 'Domain-Specific Generalization' ecosystems. The work uncovers cognitive fingerprints and offers a roadmap for next-gen embodied simulators.
OIST researchers demonstrate that AI models with 'self-mumbling' internal dialogue and working memory achieve superior learning efficiency, multitask generalization, and adaptation on sparse data. Mimicking human inner speech, the method excels in complex tasks like sequence reversal. Findings published in Neural Computation.
MIT's Tomaso Poggio likens current AI to pre-Maxwell electricity, driven by engineering without deep theory. He advocates sparse compositionality—combining simple low-dim functions—for generalization in intelligent systems. Discussion covers his learning theories from kernel machines to deep nets.
The new DM0.5 model from Yuanli Lingji has been released, trained on 150,000 hours of data. It demonstrates a 31% improvement in zero-shot performance and shows signs of emergent generalization capabilities.
Reeman releases the AI Intelligent Teaching Generalization System, granting robotic arms self-growth capabilities through intelligent demonstration learning. The system is powered by synergistic 'AI three engines' for enhanced generalization.
A Reddit discussion questions whether traditional machine learning principles still reliably guide modern practice. It highlights how rules about overfitting, model size, test-set isolation, optimizers, and ensembles have been challenged by strong empirical results.
Decoupled Descent is a proposed training method that uses approximate message passing and Onsager corrections to address data reuse bias in full-batch gradient descent. In stylized high-dimensional Gaussian mixture experiments, it provides an asymptotic certificate that training and testing errors match at each iterate.