Dear Unreal Engine developers, I have been trying to connect metahuman to chatGPT, so I could talk or write some text from UE, send it to chatGPT API and the convert the response in sound and lipsync with the metahuman.
This is only the Hello World. I will keep on working on it to improve the interactivity. I hope you like it.
This is extremely interesting. Can you share a little bit about your process? I wonder does the human have to be from metahuman? I have some models I would like to bring to life with chatgpt, started poking around to see whats possible and I’ve just stumbled on your post.
Dear friend,
This is a work in process that I am trying to finish to publish tutorial videos to show how to solve it. The main elements are: 1st a good textToSpeech with visemes information that is provided by the ReadSpeaker plugin (Expressive runtime TTS AI voice generator ) and the second is the connection to the API of chat GPT (Product )
Greetings @davidbuenov !
Splendid! Giving chatGPT an Unreal Engine digital face is a really novel idea! I imagine being able to incorporate a cinematic that begins after a question is asked…something similar to “Dr. Know” from the movie A.I. (see a clip here). I believe we have arrived to the Event Horizon of life imitating art in the blending of an A.I. language model and an interactive UI in this manner! What are your thoughts?
Looking forward to your progress with this!
I have enjoyed reviewing this fragment of AI film. I have several ideas related to this. One from the side of virtual production is to set myself inside the scene close to the metahuman and talk to her as friends, and who answer is chatGPT. The other idea is to prepare a full interface that can show the results (besides being explained by the metahuman), and there include also images, because the API allows to connect also with DALL-E. I hope to have new results soon. Thanks for your comments that push me to keep on working in this lines.
Ready to share demo project through githup or somewhere? 
Working on it to make a tutorial.
I competed in a hackathon recently where the #2 entry used ChatGPT with a very basic VR project. Mannnnn I really wanna see how you did this!
Well done! Great job!!
Hey @davidbuenov ,
How is the tutorial coming along, I am inspired to make a little project on NPC interactions and such.
However I am struggling to find useful information on how to implment your speech-to-text into GPT and then GPT’s text-to-speech right back at the player.
If you have already uploaded it elsewhere, I would be pleased if you post the link 
Dear friend, I am still on it but I can help you to try the following:
For the textToSpeech
For the Connection to OpenAI
I hope this help while I can make the video.
Thank you for the reply, I will try these out.
Can’t wait for your video, it will be a huge hit!
I am working on an audio-to-audio system with Metahumans as well. Here is a short demo and explanation of the process. It is still work in progress, but the end goal is to have a direct conversation with a character that is powered by GPT.
That’s great, we are in the same wave 
Hello there, I saw that you are working on a connection using Chat GPT and UE 5.
I am trying to do the same but using Google dialogflow for university project (well later I have to implement it in a website)
Can you tell me how to make an API request to my Google Dialogflow account ?
this is my last step, the next step is pixel streaming using a cloud computing service.
Thanks for your help
Cheers
Dear friend, I suggest you to see this tutorial to connect dialogflow with LLM.
Hope this help