So here I am trying to figure out how to remove those shadows from my normal map. It kind of looks dirty. In the ball preview within the material it looks nice and clean though.
Is there any way to keep a normal map free from shadows? I disabled cast shadow but didnt work.
Can you post an image of what you are talking about? Normal maps do not cast any form of shadows, only objects do. The only thing normal maps do is attenuate lighting response based on how the normal map faces the light.
It is possible you are seeing some kind of compression artifacts but without seeing them it is hard to tell.