I have a thin wall, which is a default cube scaled by 0.1 (10 cm) as it’s thickness (so that it looks like wall). Wall is set as BlockAll. Projectile is a static mesh with capsule, and the capsule defines the collision (and is set to block all). Projectile component is also there. While I shoot into the wall, the bullets sometimes penetrate it. Here is a video of it:
In the projectile’s properties, under the Collision category (also check Show Advanced Properties), check “Use CCD” (CCD = continuous collision detection).