It sounds like you answered your own question. There’s no right answer.
Since your asking on here I am making some assumptions, mainly that you are part of a very small team. On a larger team you would be having this type of discussion internally. So, with a really small team I think it is more important to spend your time and resources to get the actual game-play experience good. If it does well and is played a lot it would be a natural step to add matchmaking later. I think this would be very different with a larger project having an advertised release.
It’s really unique to your game. If you have a core story for what you game does, evaluate the matchmaking feature against that story. Does it support it? Does it detract from it? Or, without it will the game not meet the main story or experience you are trying to provide.