This seems to be related to explorer “overlays”. I had the Box Sync client installed which installed an overlay. When I disabled them, the crashing stopped. Since people have reported that it does it with the Helix plugin (which also uses an overlay to show file status), it seems like it is an overlay that is causing the problem.
You can disable overlays by removing the entries at:
HKLM\Software\Microsoft\Windows\CurrentVersion\Explorer\ShellIconOverlayIdentifiers