Restrict component to base class?

I have a component that should only be able to be added to classes derived from my custom pawn class, I have found the UCLASS Within specifier, but that prevents the component being added in blueprint classes. I have also tried the RestrictedToClasses specifier, but that did not seem to have any effect.