Feedback from Unreal Marketplace publishers

Roadmap for the marketplace is planned.

Awesome!

By the way since you have a plugin on the store, if you have time, I would love to hear feedback on the stability of the API regarding plugins and your experience from someone who has items on the marketplace.

I wish I have delved into some of the deeper regions of the UE4 API than I have, but I have only upgraded from 4.10 to 4.11 in both my plugin, and personal projects, but where those are concerned, I have not had anything break during the upgrade process.

As @teak421 mentioned, the first code plugins (with a couple of exceptions) have only really come out in the last week or so. It’s taken a long time for Epic to get to this stage, and there are no doubt still some issues that are holding back plugins in terms of deployment, engine version incompatibility, source code distribution, etc. Like you say, the deployment through the launcher is still very early days and is going to take a while to get to a point where it’s flexible and largely automated.

One thing I’d note - my impression is that most of the marketplace customers see plugins and blueprints very differently. There are a great many potential customers who have never used C++ but do use blueprints, and as such are more comfortable with buying blueprint assets.

I’ve no doubt there is also a common misconception that since plugins require C++ to develop, there is also some need for C++ knowledge on the part of the buyer. This isn’t the case* and I hope Epic make an effort to stress this. Having marketplace categories of ‘Blueprints’ and ‘Code Plugins’ is a bad move in my opinion - categories should be based on the content of the product, not the language it has been written in. Of course that information should be listed, but I don’t think it should be used to categorize. It is likely to scare potential buyers away from something because they see the word ‘Code’ and may think it requires coding on their part.

*Unfortunately I had a look at one of the recently released plugins and what’s required of the user for a core feature of it are an abomination. It’s disappointing that this made it through QA since it’s just going to give wary customers another reason to run away from plugins.

As for user expectations, I think it mostly depends on the type of product - for some things it’s completely reasonable to assume a level of programming knowledge, for others not. But I’d say in general, ease of use is high on most people’s lists.

API - from a C++ point of view, it has been rather unstable with a lot of breaking changes. That said, 4.10 - 4.11 was pretty seamless, so hopefully the engine is maturing and this won’t be such a big issue going forwards.

The same holds true in Unity as well (just replace Blueprints with Playmaker). I’m not sure of the numbers on either marketplace but in general at least when it comes to support tickets, I’d say a good 25% of my customers do not know how to code. I would expect the number to be higher in Unreal since its built into the engine.

At least to me this is a bit of a misnomer as well. “Code Plugins” is a bit ambiguous as to what it requires from the customer. I can as a publisher create quite a bit of C++ code for my implementation but then provide a high level blueprint for the customer to interact with that doesn’t require C++ knowledge. In general I would think this would usually be preferred and from the end user perspective it would not really be any different than a blueprint only product.

The only place that distinction is useful is where it gets installed but the user doesn’t really care about this. In the end this is also probably one of the reasons a plugin is tied to a particular engine version as well, the downloads should really be broken out to a folder that doesn’t get overwritten with a new engine install so that upgrading engine versions could not require a re download of the plugin.

I’d agree with most of what I’ve bought the quality is a bit lacking as well. I’m not sure without a large review budget that Epic can really play arbiter to quality if the marketplace takes off because there will be a TON of submission like there is currently for Unity and the reviews will usually take care of this anyways. That being said good publishers can certainly raise the bar.

This was my impression watching the substance plugin and others. Most releases there were breaking changes. I certainly don’t mind some every once in awhile especially when there is a large benefit but in general it needs to be somewhat infrequent. Quality plugins take time to make and then there are always feature request additions after launch. There is enough work to be done on those features to not pile on with calls to a core API that changed names or functions that got pulled out from under developers. The public interfaces should be well planned out most of the time so that implementation can be change/improved drastically all the time from Epic without needing to break any plugins.

Right. I don’t know, but my guess would be substantially higher, depending on the type of plugin of course.

Yeah precisely. In fact, some plugins are just editor extensions, in which case the user doesn’t even need blueprint knowledge, let alone C++.

As I understand it, the engine version limitation is an inherent C++ binary compatibility issue, so I don’t see that changing unfortunately. Which links in with the one real concern that does exist with plugins - being left hanging if the develop drops support. If the source is provided then that reduces the problem to an extent, of course.

Can you explain this a bit more? Whether you provide source or not I would think that there should always be a way to have the plugin work with any version of the engine (provided there were no breaking changes in the core API that were used by the plugin). As long as you put plugins at the right part of the build process I can’t see there needing to be anything tied to a specific version.

As I understand it, Unity doesn’t have much in the way of quality control whereas with the Unreal marketplace Epic vets each and every submission which means there is potentially a much larger amount of content which is rejected.
With that said, I don’t necessarily agree that more content = better quality/more popular.

In premise I’d agree with this, the same can be said for the IOS app store vs Android. At least looking at both marketplaces they are both in the middle currently (not fully automated but yet not as strict of guidelines as Apple). Also, both being smaller companies they tend to lean on a staff specific to the marketplace of under 10 people vs. Apple which has a much larger staff dedicated to reviews along with the budget to go along with it. Both Marketplaces certainly are smaller and can’t support a staff that is very large and when you combine this with the fact that a review of a game is much quicker than a review of a plugin. I’m not sure if you were to scale the Unreal marketplace if they can realistically properly vet everything that comes though. Some of the plugins that are already live have errors on initial load on any empty project so I’d say they already probably have a fixed amount of time they are allowed to vet an asset to keep things going through the pipeline. This just isn’t enough to really have a carefully curated marketplace.

Curating in the beginning of a store is an easy task, but as things start to scale it gets much more difficult. The 30% cut can pay for some level of curation but since both engines have gone free upfront, I would expect they are trying to allocate some of the 30% cut for other departments. Personally, I think that not over curating is probably prudent anyways since generally speaking having a brief refund window combined with reviews this doesn’t really impact customers.

This does require you to have some basic storefront items like sorting by popularity, etc. so that the junk falls to the bottom never to be seen/heard from again but I think those things become required at some scale anyways so its not like they save some budget by staying manual. Currently I’d say for those assets that actually get sold, I wouldn’t say the bar for quality is any higher currently on the Unreal Marketplace vs Unity for the coding assets (plugins or blueprints).

Precisley. If anyone at Epic in charge for the marketplace is reading this, please just call the section Plugins.

Speaking about binary compatibility, usually it’s just a question of a stable or backward compatible interface. Look at Steinberg’s VST system for audio which was simple and worked fine over decades, even today. The problem and luck we have with Unreal Engine plugins is: we can talk to nearly all modules, there might be some hundred more interfaces then in a simple audio SDK. So breaking will be unavoidable. All they can do is a deprecating scheme, so changes will not hurt immediately. :slight_smile:

From what I saw on Unity they let almost anyone submit and let the market work it out. They have a good ratings system, so good developers rise to the top and the garbage falls to the bottom. If a product has many sales and the ratings are 5/5 that is a good product. All the rest, you are taking a chance.

If doing it like that means faster submissions and bugfixes I am all for it. Get Unreal out of the way and let developers make or destroy their own reputations.

Even though Unity doesn’t have access to the source the situation is exactly the same with regards to this. The API is quite large in both engines and yes sometimes breaking changes are unavoidable and I think most people understand that. Realistically, the only thing I can speak to is the perceived frequency in which it will happen since every time it does it will require work for all plugins that use that API.

Most mature API’s the interface should really be changing very little. Regardless of what specific API we are talking about whether Global Illumination, UMG, etc. if it’s been around awhile the established interface should hopefully change very little even as updates/bug fixes get added in subsequent engine versions. Implementation can change but interface changes should really be kept to a minimum. When the substance plugin was the first on the marketplace I watched the forum threads quite a bit and it seemed that most releases of the engine broke the plugin and then Algorithmic needed to update their plugin for the new version. From the outside it didn’t see very stable and as a plugin author it did give me some pause. Granted most plugins aren’t that complex, but if every engine version or every other broke my plugins because of an API change that would be a bit frustrating. Was just trying to get others feedback for other plugins that touched some of the other APIs whether they be input, editor extensions, etc. Some are unavoidable, if it happens once a year or so that is understandable, but if it’s every release it is a bit much.

If this is the only binary incompatibility that was intended, at least from an installation perspective I don’t see why it couldn’t be moved to a different folder, so you don’t have to worry about installation again when you update to the next version of the engine.

Their review process is usually less than 5 business days. They do the normal, scan for viruses, make sure there is documentation, and make sure it doesn’t cause errors and works, and then stop there. It is mostly humans with some automation, but it is a much less manual process than it used to be. If it’s a bad product but the description doesn’t lie and it works and doesn’t hurt anything they will let it through and let the market decide things. They do have a refund policy with a small but fair window. If I had to equate it to games the review process is probably most similar to Amazon or Apple, it certainly isn’t automatic and a free for all like android or itch.io.

This does let some garbage in, and yes more content doesn’t necessarily mean better quality, but it does reduce the reviewers burden quite a bit which will help turnaround times for submission and updates. An easier/quicker process I think won’t turn off as many developers, but probably isn’t the highest priority (sales and a stable API would probably be more important). I don’t think having garbage buried at the bottom of the marketplace hurts anything either though as long as the customer is protected with a clear refund policy. Things being easier I would I think at least attract some quality developers that wouldn’t make a plugin otherwise.

I don’t know much about this stuff to be honest, but here’s some info on the issue from the man himself.

I guess there’s the question of what is meant by no breaking changes. Assuming you mean that plugin code would still recompile without alterations, and all plugin source code was distributed to buyers, then yes in that case there shouldn’t be a problem. But as Tim mentions in that post, it’s possible to modify the engine in ways that wouldn’t be breaking changes by the above definition, yet would break binary compatibility of precompiled plugin binaries. So it would come down to Epic’s policy regarding plugin source code, something which seems to have moved back and forth during the last 12 months and even now I’m not sure what they’ve settled on.

Regardless, I think @rotwang is right - the combined public interface of UE4 is just huge, so to a degree breaking changes are just inevitable. It’s going to be interesting to see how Epic approach point #2 from Tim’s post with upcoming engine releases.

Thanks for the link very informative. I didn’t see a firm decision included in that thread but assuming they went the way that Tim was leaning (requiring source and not .dlls for anything that touched the UE4 API) then I agree that there shouldn’t be an issue. I can’t see why they would deviate from this direction though since you can still have protection of anything you want to keep proprietary (even though I’m a fan of including source and do so for all of my stuff anyways). You would simply need to break out your super secret algorithm or other items into something that doesn’t touch the API which isn’t hard to do. That way all your calls to the API would have the source but anything that you wanted to hide would be in your .dll isolated from the UE4 API so no changes to it would break anything in the .dll.

This is true that some things are inevitable. Having to change things every once and awhile is no big deal. It’s all in the frequency though. If that was the only binary incompatibility and they ended up settling on the approach talked about, then I would think the only thing holding back version to version changes is the installation location. Right now it’s under …/Epic Games/(version)/Engine/Plugins which leads to friction when say you go from 4.11 to 4.12. To me this should find a different home one that doesn’t tie itself to each and every version.

Well, due to the binary incompatibility issue discussed in the linked thread, you could only have a shared installation location if the plugin were being built from source by the user. From my understanding, this generally won’t be the case. For people using the launcher version of UE4 (the majority), there is no expectation that they even have build tools installed. So my understanding would be that the plugin binaries would be built by Epic against each new engine version, and then installed to the specific engine version through the launcher.

Ideally this could be largely automated so that, assuming a plugin was not affected by breaking changes to the API on a new engine release, it would get rebuilt by Epic and be ready to go through the launcher as soon as the new engine version went live.

Ah that was the piece of information I was overlooking. I see now the distinction.

Yes this would make perfect sense to me and could certainly be automated away. Unity has something similar but for different reasons. When you are dealing with source in Unity you can do conditional compilation for platforms, editor versions, etc. so there is no issue with code but there does exist some dependencies with art. For example when going from Unity 4 to Unity 5 they changed from beast as lightmapping middleware to enlighten. Naturally these do things totally different so if you had an example you’d want the metadata that goes along with the scene (levels/maps in UE4 terms) to be different if it was Unity 4 or Unity 5. For this reason they allowed you to upload multiple versions and the user would get the correct one based on what version of Unity he is downloading from.

In the example we are talking about for Unreal you wouldn’t even need to have the publisher upload different versions since the source is provided. In the cloud they could store your plugins binaries folder for all the versions of the engine. New version comes out they can automatically compile new binaries from the source and store it in the cloud. Then when the user fires up the engine it can fetch the appropriate binaries automatically. Even if they updated mid project you could have the automation delete the old binaries and refetch the new ones probably while telling the user that it’s updating the plugins binaries.

I’ve been poking around the marketplace a lot more the last two weeks and trying to be constructive but am a bit disappointed when it comes to the code plugins. Aside from the original 5 points I had questions about, it seems from the outside one of the biggest drawbacks is everything from submission through review seems to be manual and take awhile. When you add the fact that every release will require new binaries to be uploaded this is a big drawback. There are several code plugins that I’ve purchased that don’t yet have live 4.12 versions despite having comments that said it was submitted weeks ago.

This model already seems to scale poorly and will only get worse when the amount of plugins increase. In addition to the marketplace website additions for the customer, I would think it very prudent to add a publisher portal that can handle new submissions and updates. This can only help the marketplace team by having a proper queue, things can get lost in email. Furthermore, as mentioned previously, some automation should probably be applied to the binaries problem since source is provided. It will muck up the queue every release when it doesn’t need to. I would think the majority of plugins will compile just fine and automatically make new binaries without issue, and those that don’t can auto email the publishers for them to consume whatever API has changed. This would drastically improve the queue on every major release when the marketplace starts to scale.

Manual management of marketplace updates has been a common request among sellers since the inception of the marketplace. We currently have no clue as to when it will come, or if it is still planned unfortunately.