How to structure my logic and meshes [noob]

I’m struggling to avoid having all logic inside the same class in my project. I’m trying to do some VR with a Vive which means I have two controllers inside my Pawn.
The idea was that i want to animate / keep state on the controllers, like in Tilt Brush or the intro to The Lab. Basically change materials, textures, react differently to input depending on the mode on each controller.

My first approach was to try split up the logic with raw C++ classes, which failed because I didn’t manage to bind input logic from anything that isn’t a UObject.
The next approach was to move logic to another Actor and try to use UChildActorComponent inside my Pawn… it didn’t work either, it seems UE isn’t really made for that.

Now I’m not quite sure on how to move forward without copy/paste a bunch of code for left and right controllers and bloat my Pawn.
Does anyone have any pointers on how this is supposed to be done?