Let's say you have a recording of someone's voice and the recording has the entire person's vocal range (from lowest note to highest).

Then you record your own voice saying the same thing as the other voice, making sure to say everything at the same speed at the same time or as close as you can.

Then software would compare the recordings and link the sounds together. Meaning, whenever your mic picks up a certain sound in your voice it will process it, recognize the sound, then play the sound from the other voice that was picked up when it heard that sound.

In theory this would allow anyone to use any voice while having it sound completely natural. Is this theory correct or am I missing something? If their are problems with this, what would those problems be?

3 Answers 3


This kind of technique is called corpus-based concatenative synthesis - stringing together (concatenating) little segments of sound that are pre-stored and pre-analysed (in a corpus), attempting to match an input sequence. The idea of using it with a live input is something that's being explored more and more these days. See for example Diego Constanzo's C-C-Combine, and Krotos Audio's recently released Reformer plugin. Concatenative synthesis is also a commonly used speech synthesis technique, though usually in response to eg. text input.

The problem, as user22633 pointed out, is stringing together segments in a seamless and believable way. We are so attuned to the human voice that we'd pick up on the slightest discrepancies. Besides this, building the corpus for someone's voice would be much more involved than, as you suggest, just getting a sample or two of their singing. Imagine all the possible speech segments they might articulate, in every possible pitch, not to mention glides, and differences in formant quality. Then analysis of the input would involve not only pitch tracking (a non-trivial problem in its own right) but recognition of speech segments and phonetics. Basically, your idea most likely presupposes some pretty sophisticated natural language processing techniques. That's not to say this would be impossible - especially if you put very narrow bounds on the situation, as you suggest. But without a lot of development, I'd suggest the result would be a strange robotic transformation of your voice, more than a transformation into someone else's voice. But who knows, try putting a corpus into C-C-Combine and experiment!


Stitching the selected fragments together in a manner that is indistinguishable from a seamless natural utterance is the main problem.


Not the same voice it would work a little but you tell the difference.

  • 1
    While this does technically answer the question. It would be much better if you could explain why/what parts of the idea would or wouldn't work.
    – AJ Henderson
    Jul 31, 2017 at 17:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.