🕷️
SetupAI
EN
🍎近期收集於 25h

Apple 以迭代式偽標籤改進語碼轉換 ASR

Apple 以迭代式偽標籤改進語碼轉換 ASR
PostLinkedIn
🍎閱讀原文： Apple Machine Learning
#code-switching#pseudo-labeling#speech-recognitionprogressive-refinement-cs-asrapple

💡了解迭代式偽標籤如何將未標註雙語語音轉化為更優異的語碼轉換 ASR。

⚡ 30-Second TL;DR

有什麼變化

首次將迭代式偽標籤應用於中英語碼轉換 ASR。

為什麼重要

此方法有望降低建構雙語與多語言環境中穩健語音辨識系統的成本，也為 AI 團隊利用未標註、自然產生的語碼轉換音訊提供實用途徑。

下一步行動

在未標註的中英語音語料庫上實作這套三階段流程，並與僅使用監督式資料的 ASR 基線比較詞錯誤率。

誰應關注：Researchers & Academics

關鍵要點

  • •首次將迭代式偽標籤應用於中英語碼轉換 ASR。
  • •從大型未標註語音語料庫產生偽標籤，建立半監督資料集。
  • •採用雙階段雙語模型訓練，之後進行迭代式優化。
  • •針對混合語言語句的資料稀缺與辨識複雜度提出解法。

🧠 深度解析

背景與延伸：來自公開資料，非原文內容。引用 20 個來源。

🔑 增強重點摘要

  • •Iterative Pseudo-Labeling (IPL) involves fine-tuning an existing model in successive steps, utilizing both labeled and a subset of unlabeled data in each iteration.
  • •The pseudo-labeling technique enables the creation of accurate ASR models with significantly less human-transcribed data by leveraging a model's predictions on unlabeled audio for further training.
  • •Code-switching ASR faces significant hurdles due to the inherent scarcity of mixed-language speech and text data, even when the individual languages are otherwise well-resourced.
  • •Apple previously explored a "retraining-free" code-switching ASR approach that combined existing monolingual acoustic and language models with an LSTM-based grapheme-to-phoneme model, achieving a 55.5% relative Word Error Rate (WER) reduction on intra-sentential code-switching tasks for Mandarin-English and Hindi-English.
  • •This iterative pseudo-labeling method for Mandarin-English code-switching ASR has demonstrated substantial improvements, achieving notable Mix Error Rate (MER) reductions of 6.35% on the SEAME devman subset and 8.29% on the devsge subset.
📊 競品分析▸ Show
Company/ResearchApproach/FeaturesBenchmarks/Performance (where available)
Apple (Current)Iterative pseudo-labeling for Mandarin-English code-switching ASR, leveraging unlabeled data and a two-stage bilingual training with iterative refinement.Achieved notable Mix Error Rate (MER) reductions on SEAME's devman (6.35%) and devsge (8.29%) subsets.
Meta AIExtended pseudo-labeling to multilingual settings, enabling effective ASR models using unlabeled data across 60 languages.Focus on massively multilingual ASR, not specific code-switching benchmarks provided.
GladiaSolaria-1 offers native code-switching support for over 100 languages and dialects, including high-demand BPO languages, emphasizing handling mid-sentence language changes.Claims to cover 42 languages not available from other API-level STT providers.
DeepgramSupports multilingual transcription for production speech workflows. Notes that research volume for code-switching ASR clusters around Mandarin-English, Hindi-English, and Arabic-English.Code-switched audio produces 1.5x to 11x higher error rates than monolingual baselines.
Whisper (OpenAI/Implementations)General-purpose ASR model, often used as a benchmark.Whisper-Large-v3 is an effective ASR model, though one study found LLM-based ASR systems performed better in low-resource scenarios, while Whisper performed better in Mandarin-English code-switching.
Qwen2-Audio-7B-InstructLLM-based ASR model.Showed competitive performance with 7.15% CER, 19.82% WER, and 9.76% MER on a 104-hour Mandarin-English code-switching dataset, but exhibited a tendency to overproduce Chinese text in CSASR.

🛠️ 技術深入

  • The approach consists of three main phases: pseudo-label generation, two-stage bilingual model training, and iterative improvements.
  • Pseudo-label generation: This phase utilizes existing ASR models (specifically, monolingual English, monolingual Mandarin, and an initial code-switching English-Mandarin ASR model) to generate pseudo-labels for a large corpus of unlabeled speech data.
  • Two-stage bilingual model training:
    • Pre-training: An initial bilingual ASR model, referred to as M0, is trained on semi-supervised data derived from pseudo-labeled monolingual English and Mandarin audio.
    • Fine-tuning: Following pre-training, M0 is fine-tuned on a smaller, human-annotated supervised dataset that includes both monolingual and code-switching utterances. Notably, the initial training of M0 does not incorporate any code-switching data.
  • Iterative improvements: Subsequent models (M1, M2, and so on) are progressively developed. Each iteration leverages updated pseudo-labels generated by the model from the preceding iteration, aiming to continuously refine the model's performance, particularly in recognizing complex code-switching utterances.
  • The method is designed to address common limitations in pseudo-labeling, such as confirmation bias and the accumulation of errors stemming from noisy supervision.
  • Apple's related research on "Continuous Pseudo-Labeling from the Start" (2022) introduced techniques to stabilize training by dynamically controlling the evolution of pseudo-labels through a curriculum and by sampling transcriptions from the predictive distribution rather than relying solely on the single best transcription.

🔮 前景展望AI analysis grounded in cited sources

Apple's voice assistant technologies will exhibit significantly improved accuracy and fluency in multilingual environments.
By effectively addressing code-switching challenges, the underlying ASR systems can better understand and respond to users who naturally alternate between languages.
The iterative pseudo-labeling approach will accelerate the development of high-quality ASR for other low-resource language pairs.
The method's ability to leverage large amounts of unlabeled data reduces the dependency on expensive, manually transcribed datasets, making ASR development more scalable.
This advancement will lead to more seamless and natural human-computer interaction for bilingual users across Apple's ecosystem.
Enhanced code-switching recognition directly translates to a more intuitive and less frustrating experience when interacting with devices using mixed languages.

⏳ 時間線

1992-03
Apple introduces "Casper," its first voice recognition system, with parts of its technology later integrated into PlainTalk and Siri.
2011
Apple launches Siri, the first intelligent digital assistant on a phone, significantly popularizing ASR technology.
2022-10
Apple Machine Learning publishes "Continuous Pseudo-Labeling from the Start," exploring methods to generate pseudo-labels from the beginning of ASR model training.
2026-03
Apple's "retraining-free" approach for code-switching ASR is highlighted, demonstrating a 55.5% relative WER reduction on intra-sentential Mandarin-English and Hindi-English tasks.
2026-07
Apple publishes "Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR," detailing the method for leveraging unlabeled data.
🍎閱讀原文： Apple Machine Learning
📰

AI 週報

閱讀本週精選 AI 大事摘要 →

👉相關動態

同主題

探索 #code-switching

同產品

更多關於 progressive-refinement-cs-asr

同來源

最新來自 Apple Machine Learning

AI 策展新聞聚合。所有內容版權歸原始發布者所有。
原始來源： Apple Machine Learning ↗

這是摘要，不是原文。去看原站，或訂閱每週簡報。

每週 AI 簡報

每週一封，可隨時退訂。