Yes, UE doesn’t support it. It is necessary to implement it on the App side.
You can generate the notification icon using the Android Studio or any other online service that generates the images. Add the icon in the correct folders of Build/Android/res structure.
Also, if you are using Firebase and want to receive notifications from them, it is necessary to add at the AndroidManifest.xml at Application tag: