paper

WAVENET: A GENERATIVE MODEL FOR RAW AUDIO

  • Authors:
Read the paper

📜 Abstract

This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.

✨ Summary

Summary and impact

The paper introduced WaveNet, an autoregressive raw-audio model using causal and dilated convolutions, gated activations, residual connections, and global or local conditioning. It demonstrated strong results in multi-speaker speech generation, text-to-speech, music generation, and raw-audio phoneme recognition. In its TTS experiments, WaveNet achieved mean-opinion scores above 4.0 for both North American English and Mandarin Chinese, outperforming the reported parametric and concatenative baselines. (arxiv.org)

The work directly motivated subsequent neural-vocoder research. Parallel WaveNet addressed the original model’s sequential-inference bottleneck through probability-density distillation, producing a parallel model capable of more than 20× real-time generation and deployment in Google Assistant. (arxiv.org)

The approach also transitioned into industry: Google reported that improved WaveNet systems were deployed for Google Assistant voices, and WaveNet-based voices were subsequently offered through Google Cloud Text-to-Speech. Google DeepMind later described WaveNet and related models as components of services including Assistant, Maps Navigation, Voice Search, and Cloud Text-to-Speech. (deepmind.google)