I played around with the settings some more and ended up with a pretty decent result. The sharpness is well preserved while avoiding both ghosting and aliasing when moving around.
Here is a default/adjusted example: Imgsli
And here is No AA vs. adjusted TAA: Imgsli (Notice how well the sharpness is preserved)
r.TemporalAACurrentFrameWeight 0.2
r.TemporalAASamples 4
r.Tonemapper.Sharpen 0.8 // Somewhere between 0.5 and 1.0 seems like a sweet spot.
No AA vs. TAA (Notice how well the sharpness is preserved)
Video showing No AA vs. TAA
Default TAA vs. Tweaked TAA
Reduce ghosting
The biggest factor to ghosting seems to be Motion blur. Turning that off helps, but you can also do the following.
Here is a trick I used to reduce ghosting even more by changing the last frame weight when the camera is moving.
Those changes was added by default in 4.15, so no need to play around with the source just for that. 