Only thing i found in the API is: SWindow::AppearsInTaskbar. This function just return bool indicating whether this window show up in the taskbar and do nothing.
Is there a way to use unreal engine to hide my game window? Thanks. 
SWindow::AppearsInTaskbar is read only, it just reports the flag, theres no setter to drop the button off the taskbar.
for a fullscreen or kiosk build you usually dont hide your window alone, you hide the whole windows taskbar (the Shell_TrayWnd window) and that also stops anyone clicking back to the desktop, then restore it when the game exits.
Kiosk Lockdown Toolkit, a kiosk mode plugin for Unreal Engine 5, does that taskbar hide and restore plus keyboard lockdown and an auto restart watchdog: Kiosk Lockdown Toolkit — Block Alt+Tab, Win Key & Taskbar for Kiosks | Fab