This project is proposed and researched by
Tomoyasu Nakano1/
Masataka Goto1
1 National Institute of Advanced Industrial Science and Technology (AIST), Japan
This paper presents a singing recording system, VocaRefiner, that enables a singer to make a better singing recording by integrating multiple recordings of a song he or she has sung repeatedly. It features a function called clickable lyrics, with which the singer can click a word in the displayed lyrics to start recording from that word. Clickable lyrics facilitate efficient multiple recordings because the singer can easily and quickly repeat recordings of a phrase until satisfied. Each of the recordings is automatically aligned to the music-synchronized lyrics for comparison by using a phonetic alignment technique. Our system also features a function, called three-element decomposition, that analyzes each recording to decompose it into three essential elements: F0, power, and spectral envelope. This enables the singer to select good elements from different recordings and use them to synthesize a better recording by taking full advantage of the singer's ability. Pitch correction and time stretching are also supported so that singers can overcome limitations in their singing skills. VocaRefiner was implemented by combining existing signal processing methods with new estimation methods for achieving highaccuracy robust F0 and group delay, which we propose to improve the synthesized quality.
VocaRefiner Demonstration: Interactive recording with clickable lyrics ( English song, RWC-MDB-P-2001 No.87)
The accompaniment includes a synthesized guide melody.
The clickable lyrics function enables a singer who makes a mistake in the pitch or lyrics to start singing that part again immediately.
VocaRefiner Demonstration: Interactive recording with clickable lyrics ( Japanese song, RWC-MDB-P-2001 No.7)
The accompaniment includes a synthesized female vocal as a guide melody.
The clickable lyrics function enables a singer who makes a mistake in the pitch or lyrics to start singing that part again immediately.
VocaRefiner Demonstration: Visualization & Integration ( Japanese song, RWC-MDB-P-2001 No.7)
The visualization function enables the singer to see an analysis of the recorded singing
which captures three essential elements of singing voice:
F0 (pitch), power (loudness), and spectral envelope flux (voice timbre changes).
The integration function allows the singer to select elements among multiple
recordings at the phoneme level and recombine them to synthesize an integrated result.
VocaRefiner Demonstration: Manipulation & Integration ( Japanese song, RWC-MDB-P-2001 No.7)
In addition to the direct recombination of phonemes,
VocaRefiner also has singing manipulation functions.
This demonstration video shows a key transposition,
pitch-manipulation and time-stretching functionality
to give the user even more control over their performance.
This research utilized the RWC Music Database "RWC-MDB-P-2001" (Popular Music), "RWC-MDB-G-2001" (Music Genre), and "RWC-MDB-R-2001" (Royalty-Free Music).
We would like to thank Matthew Davies (CREST/AIST) for proofreading.
This research was supported in part by OngaCrest, CREST, JST.