it could be weird loading animations async, even though its on a different thread, whats your character doing during that delay?
what id suggest is using LinkedAnimLayers to separate different anims by context and then async load the AnimLayer when its relevant
so for instance i have rifles in one layer, pistols in another, you could put swimming/climbing in others and so on