Getting address sanitizer to work in editor on Windows

Development Pipeline & Plugins
, ,
1

This question was created in reference to: [Failed to build a working address sanitizer UnrealEditor on win64 (Engine version [Content removed]

We figured out how to get this working.

Here is what we do to get an address sanitizer editor build working on Windows:

  1. Use a monolithic build.
    1. Modular builds seem to crash due to the cyclic dependencies between modules not playing nicely with ASan.
  2. Use clang as the compiler.
  3. Add the relatively new flag -fsanitize-address-outline-instrumentation to prevent inlining of instrumentation.
    1. This reduced the size of our editor executable from 2.7 GiB to 2.0 GiB.
  4. Set the target to optimize for size.
    1. This reduced the size of our editor executable again from 2.0 GiB to 1.59 GiB.

These things combined got us an instrumented editor executable under the seemingly undocumented exe size limit of ~1.85 GiB.

2

Steps to Reproduce

  1. Build an editor in monolithic mode with address sanitizer enabled.
  2. Observe that the editor fails to launch because the executable is too large.
3

Hi Chris,

Thanks for sharing the information!

Regards,

Patrick

4

Wanted to follow up and mention a few things that have changed.

Our monolithic builds are now simply too big, even when optimizing for size, to create an executable that can run on Windows. For us to run ASan, we have to go back to using a modular build.

Luckily, that clang flag -fsanitize-address-outline-instrumentation coincidentally fixes a lot of the static init issues we were encountering before. These issues typically manifest as __asan_shadow_memory_dynamic_address being null, resulting in accessing a bad address.

There was one instance that this did not address. We worked around it by disabling ASan instrumentation for that function:
FTickableObjectBase::FTickableStatics::QueueTickableObjectForAdd()

One last thing that we ran into - make sure all third party plugins are linking against the patched version of XInput.lib in the tree instead of the system one (See XInput.Build.cs). Otherwise you will hit an error on load of XInput1_4.dll.