Essentially, yes, you will need one sprite sheet for the body (that can contain the whole collection for walking, idle, fighting, taking damage etc), another sprite sheet for the headband/helmet, and then another one for pants or belts or shoes etc.
Or short, one sprite sheet for every piece, that you want to be customizable, or that you want to be able to turn on and off (to be visible or to be not visible). And yeah, to get those parts fit with the rest of the model, you have to draw them so that they animate with the rest of the sprites and are at the right position.
Here are the sheets i used in that video, i just grabbed them from an RPG Character Builder ( from this one: RPG Character Builder on Steam ) and renamed them. This should give you a rough idea, how to draw them. Especially the spear, and how its missing several parts to show, that those parts would be covered by other body parts during some of those animations.
Well, the spear seems to switch hands during walking left and walking right, but you could draw your sprites, that a weapon (or any item) stays always in the same hand, and does not switch places, if a different animation plays ^.^
RPG Sprite Char.zip (244.0 KB)
If you google for sprite sheets for such chars, then you see, that they are usually arranged in rows and coloums, where:
- a row usually contains all the sprites for one particular animation (like walking, then it contains the sprites for walking, and walking only) - at least nowadays, there is no standard, but it´s easier to arrange them in that way
- a coloum contains all the possible different animations, like walking (separated into the possible directions f.e. left, right, up, down), fighting, jumping.
And if you split a character into several pieces for customisation, then all those pieces have to be in the correct position, so that they give a correct picture, if you combine them later in your material 
And as you saw in my clip, in Unreal i only extracted Paper2D sprites for the body, since all the rest gets stacked on top of those body sprites in the material, and all textures are in the same size etc to fit together.
However, I would have to create different sprite sheets for different bodies, f.e. if i would have male and female, and demon ^.^ and then of course matching clothes for those different bodies.