I have been able to build a clean solution in a new project using 5.4, without the visual studio extension.
Currently, there is a bug which will throw an error related to Catch2 in a new unreal project when opened in VS. This bug has a quick fix available at Current Available Quick-Fix Solutions (also see UE 5.4 compile error Catch2 - #14 by Amanda.Schade). Fixing the bug using the QFE should enable us to build the solution.
My next task is to see if I can get the visual studio extension working. Its not updated and released for 5.4 yet and I have created a request for that in Developer Community. That link also has a workaround which I haven’t tried to check if it works but at this point I would rather wait for the extension to be officially released. Maybe a few votes would make it a priority 
.