Tag: #model-editing5 results
Larql decomposes static LLM models into a graph database, enabling KNN walks on layers equivalent to matrix multiplication. It allows updating internal factual knowledge by inserting data into the DB without retraining. The tool uses less memory and was created by IBM's CTO.
Open-source PickyTrain tool allows surgical editing of individual weights in GGUF models without GPU or training. Features semantic awareness for tensors like attention heads, impact warnings, drift guardrails, and full rollback. Rust core with Python bindings supports common quantizations; early release seeks contributors.
Experiments duplicating transformer layers across 6 architectures reveal a universal 'danger zone' at 50-56% depth that destroys capabilities when duplicated. Optimal duplication boosts weak models like Hybrid 9B by 75% in coding tasks. Cross-model transplants fail; models need exactly one danger zone layer.
SoLA proposes a semantic routing-based LoRA framework for lifelong model editing in LLMs, encapsulating each edit as an independent frozen LoRA module activated via semantic matching. It prevents semantic drift and catastrophic forgetting while enabling precise revocation of specific edits by removing routing keys. This is the first method to achieve reversible rollback, with end-to-end decision-making integrated into edited layers.
A Reddit discussion asks whether existing research has jointly evaluated persistent epistemic abstention, provenance preservation, evidence-triggered belief revision, and bounded collateral changes in neural systems. It seeks related terminology, partial precedents, and counterexamples across selective prediction, uncertainty estimation, belief revision, continual learning, and model editing.