Extreme Audio Time Stretching Using Neural Synthesis
Leonardo Fierro, Alec Wright, Vesa Välimäki, Matti Hämäläinen.
Companion page for a paper submitted to IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2023
Rhodes, Greece, May, 2023
The article can be downloaded here.
Abstract
A deep neural network solution for time-scale modification (TSM) focused on large stretching factors is proposed, targeting environmental sounds. Traditional TSM artifacts such as transient smearing, loss of presence, and phasiness are heavily accentuated and cause poor audio quality when the TSM factor is four or larger. The weakness of established TSM methods, often based on a phase vocoder structure, lies in the poor description and scaling of the transient and noise components, or nuances, of a sound. Our novel solution combines a sines-transients-noise decomposition with an independent WaveNet synthesizer to provide a better description of the noise component and an improve sound quality for large stretching factors. Results of a subjective listening test against four other TSM algorithms are reported, showing the proposed method to be often superior. The proposed method is stereo compatible and has a wide range of applications related to the slow motion of media content.
Listening Test Sounds
These are the sounds used in the listening test described in the paper.
|
|
Original |
HP-TSM |
Fuzzy PV |
IPL-PV |
|
| Soda 4x |
|
|
|
|
|
| Fireworks 4x |
|
|
|
|
|
| PingPong 4x |
|
|
|
|
|
| Sneeze 4x |
|
|
|
|
|
| Saw 4x |
|
|
|
|
|
| Rooster 4x |
|
|
|
|
|
|
|
Original |
HP-TSM |
Fuzzy PV |
IPL-PV |
Proposed |
| Soda 8x |
|
|
|
|
|
| Fireworks 8x |
|
|
|
|
|
| PingPong 8x |
|
|
|
|
|
| Sneeze 8x |
|
|
|
|
|
| Saw 8x |
|
|
|
|
|
| Rooster 8x |
|
|
|
|
|