You would need to create UProperty class (look up engine source code to see how they are implmented) which represets the property type:
also you need to modify UHT to make it recognize it
and ofcorse Slate UI if you want to make it apper in property editor and you need to do the same with node pins:
I never even think about doing that so this is all i can give you, it might need even deeper modifications in core so it might be painful. Note that you can create custom property UI for any type that reflection system supports for example custom UI specific structure type (good example here is FSlateBrush), so if you aim for that you don’t need to create custom property type for just that