So I finally made it work. The previous post is part of the solution (thanks a lot for that!), but it was not sufficient in my case. On top of what is outlined above, I realized that python was giving me warnings when installing tensorboard, namely that
C:/Program Files/Epic Games/UE_5.4/Engine/Binaries/ThirdParty/Python3/Win64/Scripts
was not in my PATH environment variable.
Which was weird, because… it was.
The problem: whitespaces. Path variables don’t always work when there are whitespaces in them (but they apparently sometimes do… as i said in my original post, the editor had no trouble finding tensorboard). The standard path to the engine on many windows machines (including mine), and also the one which is used in the LearningToDrive tutorial, is
C:/Program Files/Epic Games/UE_5.4/Engine
which includes a whitespace in “Epic Games” (the space in “Program Files” is somehow okay though… 
). Combined with the solutions outlined in the previous post, changing the engine installation folder to a path which does not have any whitespaces removed the tensorboard installation warning, and LearningAgents finally managed to find it.