Detail publikace
Target Speaker ASR with Whisper
POLOK, A. KLEMENT, D. WIESNER, M. KHUDANPUR, S. ČERNOCKÝ, J. BURGET, L.
Originální název
Target Speaker ASR with Whisper
Typ
článek ve sborníku mimo WoS a Scopus
Jazyk
angličtina
Originální abstrakt
We propose a novel approach to enable the use of large, single-speaker ASR models, such as Whisper, for target speaker ASR. The key claim of this method is that it is much easier to model relative differences among speakers by learning to condition on frame-level diarization outputs than to learn the space of all speaker embeddings. We find that adding even a single bias term per diarization output type before the first transformer block can transform single-speaker ASR models into target-speaker ASR models. Our approach also supports speaker-attributed ASR by sequentially generating transcripts for each speaker in a diarization output. This simplified method outperforms baseline speech separation and diarization cascade by 12.9% absolute ORC-WER on the NOTSOFAR-1 dataset.
Klíčová slova
target-speaker ASR, diarization conditioning, multi-speaker ASR, Whisper
Autoři
POLOK, A.; KLEMENT, D.; WIESNER, M.; KHUDANPUR, S.; ČERNOCKÝ, J.; BURGET, L.
Vydáno
6. 5. 2025
Nakladatel
IEEE Biometric Council
Místo
Hyderabad
ISBN
979-8-3503-6874-1
Kniha
Proceedings of ICASSP 2025
Strany od
1
Strany do
5
Strany počet
5
URL
BibTex
@inproceedings{BUT198049,
author="Alexander {Polok} and Dominik {Klement} and Matthew {Wiesner} and Sanjeev {Khudanpur} and Jan {Černocký} and Lukáš {Burget}",
title="Target Speaker ASR with Whisper",
booktitle="Proceedings of ICASSP 2025",
year="2025",
pages="1--5",
publisher="IEEE Biometric Council",
address="Hyderabad",
doi="10.1109/ICASSP49660.2025.10887683",
isbn="979-8-3503-6874-1",
url="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10887683"
}Dokumenty