Help with Geometry Collections as walls

Hello,

I posted about earlier today, but I am really stuck here:

I want to have a maze with explodable walls (Geometry Collections), yet my player is always able to push it around. How can I create it so that my player only collides with the Geometry Collections, but not pushes them, while an explosion does break them?

edit:
Locking the physics constraints does not seem to work

Bombcode: