We have an outage on answers.unrealengine.com yesterday that affected a small number of users. Users was unable to login to answers.unrealengine.com. Users that have ‘remember me’ checked are not impacted.
We apologize for the inconvenience this may have caused. There are efforts in motion to make sure that this doesn’t happen again.
We have an authentication system that is used by all our websites, including unrealengine.com, forums.unrealengine.com, wiki.unrealengine.com, andanswers.unrealengine.com.
We engaged external vendor to help us build and maintain answers.unrealengine.com.
Due to lack of communication on our side, our external vendor was not notified of the changes that we made to authentication service. Hence, no proper testing was done.
We identified the problem, and have since worked with our external vendor to make sure the proper communication channel is setup properly.
On the bright side, we just made our code more resilient
Below are the details of what happened:
11 June early afternoon PST: we released a number of backend server code, which includes our authentication service code.
12 June early morning PST: we received report that some users failed to login.
Upon investigation,we found out the root cause at 6.31 AM PST, and pushed out the fix at 6.44 AM PST.
For those technical folks out there:
The issue is caused by us making changes to return more attributes on the login call.
Our code on answers.unrealengine.com did a strict de-marshalling, hence failed on new attributes.
We fixed this by making de-marshalling to be less strict to help with backward compatibility and avoid issues like this from happening.