I think there is a couple of mistake here, first you do transforms from screen space to world space, it should be the opposite, also you divide by the normals length in the end, but normals in UE4 are automatically normalized, so they should be equal to one. I am no GLSL expert but in the original code it was referring to vertex, due to the nature of the algorithm dividing by distance to vertices or even pixels could make sense but I am not sure.