Tokyo U. launches medical Japanese LLM

Qwen2.5 Base Architecture (applicable to 109B variant):

• Decoder-only Transformer backbone with rotary positional embeddings (RoPE) and QKV bias for length generalization[2]
• Grouped Query Attention (GQA) with multiple query heads and reduced key/value heads for improved cache utilization[2]
• SwiGLU-activated feed-forward networks with dimensionality optimization[2]
• Pre-normalization with RMSNorm for training stability[2]

Performance Characteristics:

• Context window: Full 131,072 tokens with 8,192 token generation capability[5]
• Qwen2.5-72B-Instruct achieves 86.6 on HumanEval and 88.2 on MBPP coding benchmarks, indicating strong reasoning capabilities transferable to medical domain[3]
• Quantization support (8-bit and 4-bit via GPTQ) enables deployment flexibility with up to 2.5× throughput improvements[2]

Medical Domain Optimization:

• Predecessor Med-Qwen2-7B demonstrates improved accuracy in medical document analysis, diagnosis support, and specialized medical text generation[3]
• Qwen2.5-Math variant employs Chain-of-Thought (CoT) and Tool-Integrated Reasoning (TIR) for complex multi-step problems, applicable to medical reasoning tasks[4]