Figured it out based on the unreal source code for 4.25 on github.
In OnlineSessionAsyncServerSteam.cpp, change GetPublicIP() to GetPublicIP().m_unIPv4 in order to retrieve the uint32 value from the SteamIPAddress_t struct, at lines 410 and 446:
NewSessionInfo->HostAddr->SetIp(SteamGameServerPtr->GetPublicIP().m_unIPv4);
SteamUser()->AdvertiseGame(k_steamIDNonSteamGS, SteamGameServerPtr->GetPublicIP().m_unIPv4, Subsystem->GetGameServerGamePort());
Compiled successfully after that