Found the solution for the problem and I feel really dumb. If anyone encounters the same issue, here is something you can try.
https://github.com/maluoi/openxr-explorer
Setup the openxr-explorer and change the runtime to steam-vr. I though I can skip this step because it worked all the time.