A Diffusion-Based Generative Equalizer for Music Restoration
Eloi Moliner, Maija Turunen, Filip Elvander and Vesa Välimäki
Companion page for a conference paper submitted to International Conference on Digital Audio Effects: DAFx-24
Trained models available at: here.
Abstract
This paper presents an innovative approach to audio restoration, focusing on the enhancement of low-quality music recordings, and in particular historical ones. Building upon a previous algorithm called BABE, or Blind Audio Bandwidth Extension, we introduce BABE-2, which presents a series of significant improvements. This research extends the concept of bandwidth extension to generative equalization, a novel task that, to the best of our knowledge, has not been explicitly addressed in previous studies. BABE-2 is built around an optimization algorithm utilizing priors from diffusion models, which are trained or fine-tuned using a curated set of high-quality music tracks. The algorithm simultaneously performs two critical tasks: estimation of the filter degradation's magnitude response and regeneration of the restored audio. The effectiveness of the approach is objectively evaluated on historical piano recordings, showing a marked enhancement over the prior version, and similarly yields impressive results in rejuvenating the works of renowned vocalists Enrico Caruso and Nellie Melba. This research represents a significant advancement in the practical restoration of historical music.
Audio Restoration Examples
Jewel Aria: Nellie Melba
| Title | ID | Performer | Composer | Year |
|---|---|---|---|---|
| "Air des bijoux (Jewel Song)" | Victor (88066) | Nellie Melba | Gounoud | 1910 |
Reference singer: Female-1 from VocalSet:
Jewel Aria: Adelina Patti
| Title | ID | Performer | Composer | Year |
|---|---|---|---|---|
| "Air des bijoux (Jewel Song)" | G&T (3056) | Adelina Patti | Gounoud | 1905 |
Reference singer: Female-5 from VocalSet:
Vesti la Giubba: Enrico Caruso
| Title | ID | Performer | Composer | Year |
|---|---|---|---|---|
| "Pagliacci - Vesti la giubba (Act 1)" | Victor (6001-A) | Enrico Caruso | Leoncavallo | 1910 |
Reference singer: Male-2 from VocalSet:
Vesti la Giubba: Beniamo Gigli
| Title | ID | Performer | Composer | Year |
|---|---|---|---|---|
| "Pagliacci - Vesti la giubba (Act 1)" | Victrola (643-B) | Beniamo Giglie | Leoncavallo | 1922 |
Reference singer: Male-11 from VocalSet:
More Audio Restoration Examples (used in the evaluation)
Piano Recordings
The complete evaluation data can be accessed here here.
_78_etude-in-f-minor_sergei-rachmaninoff-e-donagnyi_gbia0187069a_EtudeinFMinor-SergeiRachmaninoff-E.Donagnyi
78_waltz-in-c-sharp-minor_leopold-godowsky-chopin_gbia0299517a_WALTZINCSHARPMINOR-LEOPOLDGODOWSKY
_78_cradle-song_percy-grainger-brahms-percy-grainger_gbia0322432a_CRADLESONG-PERCYGRAINGER-Brahms-PercyGrainger
_78_edelweiss_victor-young-albert-jungmann_gbia0191886a_EDELWEISS-VICTORYOUNG-AlbertJungmann
_78_hungarian-rhapsody-no-8_mark-hambourg-liszt_gbia7000660b_HUNGARIANRHAPSODY_No.8-MARKHAMBOURG
Singing Voice Recordings: Enrico Caruso
The complete evaluation data can be accessed here here.
_78_siciliana_sig-enrico-caruso-mascagni_gbia0518643a_Siciliana_-_Sig._Enrico_Caruso_-_Mascagni
_78_andrea-chenier---come-un-bel-di-di-maggio-as-some-soft-day-in-may-act-4_enrico_gbia0012555a_Andrea_Chenier_-_Come_un_bel_di_di_maggio__-_Enrico_Caruso
_78_amor-mio_enrico-caruso-ricciardi_gbia0077693a_Amor_Mio_-_Enrico_Caruso_-_Ricciardi
_78_la-danza-tarentella-napolitana_enrico-caruso-pepoli-rossini_gbia0479696a_La_Danza-Tarentella_Napolitana_-_Enrico_Caruso.flac_denoised_vocals.wav_cropped
Singing Voice Recordings: Nellie Melba
The complete evaluation data can be accessed here here.
_78_annie-laurie_dame-nellie-melba-douglas-scott_gbia0243779a_Annie_Laurie_-_Dame_Nellie_Melba_-_Douglas
_78_nozze-di-figarovoi-che-sapete-marriage-of-figarowhat-is-this-feeling_dame-nell_gbia7028616b_Nozze_di_Figaro_Voi_che_sapete__Marria_-_Dame_Nellie_Melba
_78_songs-my-mother-taught-me-als-die-alte-mutter_nellie-melba-dvok_gbia7026919a_Songs_My_Mother_Taught_Me__Als_die_alte_Mut_-_Nellie_Melba
_78_hamletophelias-mad-scene-scne-et-air-dophlie_dame-nellie-melba-thomas_gbia7028613b_Hamlet_Ophelia_s_Mad_Scene__Sc_ne_et_a_-_Dame_Nellie_Melba