What is the actual end class of your Primitive Component class hierarchy? Is it Brush/BSP, Static Mesh, Skeletal ?
I dont think primitive component is something you use directly, you need at least a box component, sphere component, etc.
No accessing the vertices is not expensive, at least not until you start looping over 50k or 100k of them. But if you break up the loop you’ll still be fine 