If it’s single player you could do something like add a box collision on the side he should be able to move through. Once he touches that one the collision is turned off until he left both collisions (your box collision and mesh / original box collision). Then once that happened turn it back on.
For multiplayer it’s a bit more funny. I have no good idea from the top of my head how that could be done.