OpenAI, the research lab behind the generative artificial intelligence tool ChatGPT, has unveiled its new voice-cloning technology, which it calls “Voice Engine.” This audio model can create a synthetic copy of a person’s voice, accent, and other human speech patterns based on a 15-second audio sample.
“Today we are sharing preliminary insights and results from a small-scale preview of a model called Voice Engine, which uses text input and a single 15-second audio sample to generate natural-sounding speech that closely resembles the original speaker. It is notable that a small model with a single 15-second sample can create emotive and realistic voices.” OpenAI says in its blog post.
Voice Engine has been privately tested with OpenAI’s trusted partners to better understand how the tool could be used for good across different industries. In one example, the voice of a young patient, who experienced significant loss of speech due to a vascular brain tumour, was replicated using a previous recording she made for a school project.
The speech synthesis platform has also been tested to provide reading assistance to non-readers and children and to translate content into other languages, like videos and podcasts, allowing content creators and businesses to connect with more people around the world.
According to OpenAI, Voice Engine was developed in late 2022 and has been powering the text-to-speech API as well as ChatGPT Voice and Read Aloud.
While Voice Engine presents many uses and benefits, OpenAI recognises the risks associated with the technology and the “potential for synthetic voice misuse.“
”We hope to start a dialogue on the responsible deployment of synthetic voices and how society can adapt to these new capabilities,” OpenAI stated, acknowledging the widely criticised deepfake manipulation, where malicious actors could exploit voice synthesis to impersonate individuals, such as political leaders or corporate executives, disseminating false information or committing financial scams.
The company is not yet releasing the model to the public and continues to have discussions regarding the potential deployment of the technology at scale.
Earlier this year, OpenAI also introduced its AI model, Sora, a text-to-video generator that can create high-quality and hyper-realistic scenes up to a minute long based on text instructions.