I wasn’t able to include the file directly from the Engine directory, however, adding to the [ProjectName].build.cs file in this way allowed me to use the files associated with the OnlineSubsystemUtils plugin found in the engine directory.
I added this line to my build file:
PrivateDependencyModuleNames.AddRange(new string[] { "OnlineSubsystemUtils" });
I was then able to include what I wanted without having to reference anything but the header of the file/plugin file I wanted to use. I know this is late, but maybe it will help someone.