[FREE] SpeechGPT: Example project mixing OpenAI GPT and Microsoft Azure Speech Services

Community Showcase
, , ,
1

image
image1920×1031 32.8 KB

Example Project that uses both OpenAI GPT and Microsoft Azure Speech Services

Links

Requirements

  • Visual Studio 2019 or 2022 with the Module: Game Development with C++
  • Unreal Engine 5.1
  • Git with Git LFS

Installation

  • Run these commands in Git Bash:
git clone https://github.com/lucoiso/UESpeechGPT.git
cd UESpeechGPT
git submodule update --init --recursive

Support me: Sponsor @lucoiso on GitHub Sponsors :heart:

6 Likes
2

New Additions:

  • Speech to Text
  • Multiple Languages
  • Language Identification
  • Icons

image
image1920×1031 27.5 KB

Check it out on my github! : )

1 Like
3

REALLY COOL THANKS

1 Like
4

Created a new post for HttpGPT! : )

1 Like
5

Hello, i have to say its really great.I download the zip from githob.However,when i generate the project,it shows the message ERROR: Expecting to find a type to be declared in a target rules named ‘SpeechGPTTarget’. This type must derive from the ‘TargetRules’ type defined by Unreal Build Tool.Is there anything i need to set up for the project?

6

Hi! Have you downloaded the project from the Release page? SpeechGPT Releases (github.com)

This button below doesn’t work well with Unreal projects : /
image

Use this link to download the Project: Releases · lucoiso/UESpeechGPT (github.com)

7

SpeechGPT v1.1.0

image
image1920×1031 75.5 KB

What’s Changed

  • Now using the new Chat API
  • More settings, delegates & logs
  • Chat history
  • New options

Full Changelog: Comparing v1.0.0...v1.1.0 · lucoiso/UESpeechGPT · GitHub

9

SpeechGPT v1.1.1

Changes

Update AzSpeech to v1.5.0

  • Fix error logging
  • Add new setting to enable usage of private endpoints
  • Set audio compression to not occur in transient sound waves
  • Fix audio compression occurring more than one time
  • Fix audio compression being called outside audio thread
  • Separate tasks signals to avoid the completion delegate sending wrong data (this fixes the ‘sound wave with 0 duration
’ issue)
  • Set to only broadcasts be called in game thread
  • Set the runnable threads to try lock on exit to avoid crashes after End PIE
  • Add comments in some functions
  • Move Viseme Data to a separate header
  • Create new structure: Animation Data
  • Added new functions to extract Animation JSON Data from Viseme Data

Update HttpGPT to v1.2.0

  • Create new structures: Options & Error
  • Set all tasks to receive an options param
  • Set settings to contain a default options object
  • Set the response to contain the error object if bSuccess is false
  • Transform the role param in an Enumeration
  • Set the message body to use the options or the default options if no one was given to the task. Also allowing the usage of other models.
  • Set the plugin to contains 2 functions to manage an entire chat or send a single message.
10

I get the error “Last Status: request not sent” for the 1.1.1 SpeechGPT example project when trying to sent prompts. What am I doing wrong?

11

SpeechGPT v1.2.0

Changes

  • Add settings
  • Add system context
  • Add support for streamed response

Modules Updated

Screenshots

image
image1920×1031 32.8 KB

12

SpeechGPT v1.2.1

Changes

Now including the new Editor Tool: HttpGPT Chat

Editor
Editor1920×1080 351 KB

13

Hellow. I have download the latest V1.2.1 to setup, but there is still some problem. Could you help me how to fix it.

Running C:/UE_5.0/Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.exe -projectfiles -project=“C:/Users/Render_2/UESpeechGPT/SpeechGPT.uproject” -game -rocket -progress -log=“C:\Users\Render_2\UESpeechGPT/Saved/Logs/UnrealVersionSelector-2023.03.22-18.06.20.log”
Log file: C:\Users\Render_2\UESpeechGPT\Saved\Logs\UnrealVersionSelector-2023.03.22-18.06.20.log
Log file: C:\Users\Render_2\AppData\Local\UnrealBuildTool\Log_GPF.txt

Some Platforms were skipped due to invalid SDK setup: IOS, Android, Linux, LinuxArm64.
See the log file for detailed information

Discovering modules, targets and source code for project

C:\Users\Render_2\UESpeechGPT\Source\SpeechGPT.Target.cs(14,3): error CS0103: The name ‘IncludeOrderVersion’ does not exist in the current context
C:\Users\Render_2\UESpeechGPT\Source\SpeechGPT.Target.cs(14,25): error CS0103: The name ‘EngineIncludeOrderVersion’ does not exist in the current context
C:\Users\Render_2\UESpeechGPT\Source\SpeechGPTEditor.Target.cs(14,3): error CS0103: The name ‘IncludeOrderVersion’ does not exist in the current context
C:\Users\Render_2\UESpeechGPT\Source\SpeechGPTEditor.Target.cs(14,25): error CS0103: The name ‘EngineIncludeOrderVersion’ does not exist in the current context
ERROR: Expecting to find a type to be declared in a target rules named ‘SpeechGPTTarget’. This type must derive from the ‘TargetRules’ type defined by Unreal Build Tool.

1 Like
14

Hi! : )

The errors are related to IncludeOrderVersion which was added in UE5.1.

This project was built on UE5.1, so it won’t works well on UE5.0. Even if remove this line from the Target.build.cs, the assets will not be loaded 'cause was created in a newer version of the engine. : /

15

thanks. I will try again.

1 Like
17

I am more of an artist than a programmer and have run into some issues that you may be able to help with.

I downloaded the plugin through the Unreal marketplace and the Speech GPT example project through git. On opening I was asked to rebuild plugins. This failed on the GPT plugin. I made a new project and copied over the content and plugins. This allowed me to open the project and for the most part everything is working.
In the UI I have some errors with missing variables. I tried recreating the nodes but those variables don’t exist.
I am running UE 5.1

1 Like
18

I sorted this out by using Mak AZSpeechRecognition options

1 Like
19

I think I see the issue with AzSpeech plugin 1.6.5 now. I am using the Marketplace version, and deleted the version inside the Plugins folder to make that work properly. Once the plugin is enabled in Settings and the keys are set, I had to refresh the nodes, which removed many of the links. However, because the variables are now set in the Settings, we don’t need them set in the Blueprint anymore, so they work anyway, and the whole level just works. So, the application is working again, as expected, in a way that is unexpected.

Next stop is to try my customizations. This is such a great set of plugins!

1 Like
20

Apparently the latest AzSpeech plugin update (1.6.5) has changed the parameters for AzSpeechSettingsOptions, and the Blueprint graphs no longer work. I’m not enough of a developer to fix that on my own. It took away pins for Profanity Filter, Private Endpoint, Region ID, and Subscription Key. When you Refresh Node, the node that is left doesn’t have any useful options, just objects that I cannot find any way to leverage.
This is such a great and useful plugin and demo. I can’t thank you enough for it - it’s been very educational for me. I’m sorry if I don’t understand the details of making it work, but I am trying.

2 Likes
21

Hi. I can not use this chat, cause I have this error message:

image
image1379×221 8.88 KB

Any advice?

1 Like
22

Hi! Recently updated the .zip file in the latest release: SpeechGPT v1.2.1 Latest (github.com)

SpeechGPT v1.2.1

Including the new Editor Tool: HttpGPT Chat

image
image1920×1080 295 KB

Including the new Editor Tool: HttpGPT Image Generator

image
image1920×1080 300 KB