It only happens occasionally with mouse for me (right clicking works for dragging most of time, then sometimes it freaks out, usually after creating a new node). It happens every single time with trackpad for me, just as you described. I assume (possibly mistakenly) that fixing trackpad issue will resolve mouse variant.