I have a similar issue, but there is no collision after packaging if I use Set Start and End from Blueprints in game, e.g. after a 0.1s delay after Begin Play.
Update: Spline Mesh collision also doesn’t work when a spline mesh is spawned at runtime! I’m testing this now and I’ll make a bug report soon. I need to see if this also happens on earlier versions (I’m on 4.24).
Update 2: It looks like a bug in 4.24+, in 4.23 it works properly. I’ve reproduced this and reported through the bug submission form.
Here’s a simple repro project if you’re interesed:
https://app.box.com/s/6fo9ysf38r7vd8tgvjvjee6jdnh7ejeb
… Works ok in PIE, but the sphere falls through after packaging.