Using Epic's Content on Marketplace as Starting Point for Project.

Are the project examples that Epic releases “free game” for a developer to take as a sort of starting point and develop a game from that? I’ve had a hard time finding out if all the examples are limited to being examples only or if we can use them.

You can use them as a starting point. The examples are free to use, you cannot however make an example yourself with those examples and release it on the marketplace. You can however release a game with said assets.

As long as you use them within UE4, you can’t use those assets with another engine.

Actually no. You can use Epic assets in your own assets pack. Assuming you are not just repacking them and trying to sell.
But they can be complementary.