Best way to do vertex displacement on meshes with different materials

Hey guys, I have a low poly fish where different surfaces have different materials, i.e. body, eye and so on. That means the fish has 5 materials instead of only one. I’d like to animate the fish using vertex displacement / world position offset.
Do I need to do that for every material of the fish or is there a better way? Can I have like a master material for the whole static mesh of the fish?