The comments above really helped, and after scanning the source code for a bit I figured out the expected tags.
As stated above, you need to create a Text Style Sheet with the row type of “Rich Text Style Row”, which then needs to have a “Default” row. Any additional styles can then be added, in my case a “Cool” row that appears blue.
And then you need to tag your input string with " < Style Name> Your Text </>". In my case this was “Hello World</>”.
For the images, it’s a similar process. You’ll need a class that inherits from RichTextBlockImageDecorator, which will need a reference to a data table that uses the “Rich Image Row” row type.
Here you can create a named brush that you can reference in your string. In my case it’s “Face”.
In your input string, you can get that image with < img id=“Face” />.
Hope this prevents someone from spending hours digging through code. It was a really easy thing to spot too, but I had missed it my first time around.