Apparently after a lot of headaches the issue was the map itself, so leaving this answer here for the Blender users: the map I used was baked in Cycles and produced the mentioned issues, whereas the maps baked in Blender Internal worked perfectly. I’m yet to look into if I could somehow salvage Cycles maps instead of rebaking everything. I hope this helps anyone out!