By further experimenting we found this happens in particular of ps4 Controllers (not on xbox controllers)