Neural voices compared with older robotic voices

Older synthetic voices build speech from pieces or from rules. Newer neural voices learn from recordings and predict the sound. That difference explains much of what you hear. It also tells you what to check before you pick one for a phone message, a video or a training course.

The older methods

The source describes two main older routes.¹

Concatenative synthesis strings together short segments of recorded speech. Done well it's generally the most natural-sounding of the older kind, but where the joins don't match, you can hear audible glitches.¹ Another description of it says the result can sound unnatural, with an odd cadence and tone.²

Formant synthesis uses no recordings at runtime. It calculates a waveform from an acoustic model. Many systems built this way sound robotic and would never be mistaken for a person. They do have uses: they're usually smaller programs than concatenative ones, so they can be used in devices where memory and processing power are especially limited.¹

A third kind, diphone synthesis, keeps one recorded example of each sound-to-sound transition in a language. It suffers from the glitches of concatenative synthesis and the robotic sound of formant synthesis, and its use in commercial products is declining.¹

So "robotic" isn't a flaw in one product. It's often a result of how the voice is made. Some voices sound flat for exactly that reason.

What neural voices do differently

A neural voice uses a deep neural network, a kind of computer model that learns patterns from examples. For speech, it's trained on a large amount of recorded speech together with the matching text.¹

WaveNet, an early and well-known example, was trained with recordings of real speech. Even so, when it appeared in 2016 its output was still reported as less convincing than actual human speech.² Voices have kept changing since, so treat that as history, not a current rating.

Why the choice shows up in tools

Many cloud tools list the voice type as a setting. Amazon Polly's documentation, for instance, names four engines: standard, neural, long-form and generative.³ The documentation tells you to provide an engine that is supported by the voice you select, so the engine and the voice go together.³

Check three things in the maker's own documentation:

• Which engines or voice types your chosen voice supports.
• Whether the price differs by type. Prices change, so note the date you looked.
• Whether the markup you plan to use, such as SSML, works with that type.

What even good voices still get wrong

Rhythm and emphasis are hard. The source says prosody, which means pitch, timing and stress, remains a challenge and an active research topic.¹ A neural voice can still put the stress on the wrong word or read a name oddly.

That's why a listening check matters whatever voice type you choose. Don't skip it. For phone messages and training audio, play the whole file once, start to end, before it goes live. Fixes for stumbles are in fixing names and terms a voice gets wrong.