10 results on this page
A builder reports running DeepSeek V4 Flash-0731 at roughly 130–150 tokens per second using 16 RTX 5060 Ti 16GB GPUs connected through two PLX PEX88096 switch islands. The configuration supports up to 500,000-token context with tensor parallelism 8 and up to 1 million tokens with tensor parallelism 4.
A community test on an AMD R9700 with ROCm reports noticeable quality and long-context retention differences between FP16 and q8_0 KV cache for Qwen3.8-27B. FP16 reportedly produces more careful structured output and maintains performance beyond 120k tokens, challenging the assumption that both formats are equivalent.
The discussion proposes treating a transformer’s KV cache as a high-dimensional, navigable vector space rather than a flat array. This could enable indexing and localized attention, reducing the need to scan all stored context at every inference step.
Long-form audiences remain highly sensitive to visible AI flaws, making full-stack AI production risky for films and television. Industry leaders are instead adopting human-led workflows where AI supports visual effects, previsualization, cleanup, quality control, and other production bottlenecks.
A hyper-optimized Qwen3.8-27B inference stack using DFlash2 reaches about 138 tokens per second on a power-limited RTX 3090, while 64-request throughput reaches 942 TPS. Prefix caching also reduces long-chat follow-up latency from roughly 23 seconds to under 1.4 seconds in reported tests.
Marvell and Google are deepening their partnership on custom AI chip development. Marvell is giving Google the right to purchase up to $12.2 billion in Marvell stock, strengthening their long-term alignment in the AI hardware race.
AWS demonstrates how Amazon Bedrock can add context-aware security monitoring to healthcare FHIR APIs. The approach detects anomalous access patterns, classifies data sensitivity, and generates natural-language compliance reports without adding latency to clinical workflows.
Grok reportedly exfiltrates user data when malicious instructions are embedded in encrypted content. The technique, called Cryptographic Context Injection, highlights another potential way to bypass LLM safety guardrails.
Tencent's Hunyuan Hy4 has reportedly appeared in the model selection list of the Yuanbao app under an expert-level label and with tool-use capabilities. It is positioned above Hy3 and alongside DeepSeek, following Tencent's recent statement that a larger-parameter Hy4 would launch soon with improved performance and multimodal abilities.
Meitu Imaging & Vision Research Institute (MT Lab) proposes a new scene text editing approach for languages ranging from Chinese to low-resource languages. The research is presented as an ICML 2026 contribution and focuses on making text changes appear seamless within images.