problem with water on Raised areas

I’ve got rivers and lakes on raised plateaus and for some reason the water keeps forming this around the base of the cliffs like this. I know it has something to do with the rivers because when I delete the rivers but leave the lakes it the water on the cliffs disappear. Anyone know how to fix this?