It sounds really complicated indeed, but it’s an interesting approach for sure.
This is great, in my opinion it is details like that making a game feel really immersive. Even though the players may not even notice it consciously, they will unconsciously. I think attention to detail always pays off.