Do I have to handle manually memory allocation for custom classes?

So, up to this point I worked only with classes derived from UObject and AActor, but now I’m moving towards composition stuff and my own little “Components” are simple classes, not derived from UObject or any UE4 classes.
They are extremely primitive and basicaly they have variables, calculate stuff and behaviour and then pass data into actual UE4 components.
My question is - Do I have to handle C++ memory allocation part by myself, because UE4 GC working only with classes derived from UE4 classes?

IF you are using anything else than UObjects, you have to manage memory manually.

Though smart pointers, should make it bit easier.
Like TSharedPtr<>, TWeakPtr<>, TAssetPtr<> etc. There are lots of them.
Though if coruse those present it’s own challenges like circular references.