Ok so I looked into how the engine handles the messages
A simple example
FMessageDialog::Open(EAppMsgType::Ok, FText::FromString("Error"), FText::FromString("Error title"));
There are also confirm / cancels but they are more complex.
I’ll do a packing test with it.
Edit: Can confirm it works in packed shipping. It pops up as a native message box.
Can’t capture the shipped version as it steals focus from the windows capture and is in fullscreen.