Issues when compiling very large UWP applications

Issues when compiling very large UWP applications

Introduction

When compiling very big UWP applications, you can get the following error:

C:\Program Files (x86)\Microsoft SDKs\UWPNuGetPackages\microsoft.net.native.compiler\2.2.8-rel-28605-00\tools\Microsoft.NetNative.targets(801,5): error : ILT0014: Failed to compile interop source code. See the build log for error details.

This is caused by the compiler going out of memory.

64-bit compiler

One possible solution is to use the 64-bit compiler, that incredibly is not very well documented. The only reference that I have found is in a comment for a GitHub issue when compiling Xamarin.Forms apps for UWP.
Essentially, in the application settings, add the Use64BitCompiler setting:

<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
  ...
  <UseDotNetNativeToolchain>true</UseDotNetNativeToolchain>
  <Use64BitCompiler>true</Use64BitCompiler>
<PropertyGroup />
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
  ...
  <UseDotNetNativeToolchain>true</UseDotNetNativeToolchain>
  <Use64BitCompiler>true</Use64BitCompiler>
<PropertyGroup />

This will invoke the 64-bit version of nutc_driver.exe (the Native UTC compiler compiler), so it will be able to use much more memory, and generally it will solve the compiler issues.

Non 64-bit toolchain!

In reality, the above flag invokes the 64-bit compiler, but not all the toolchain is 64-bit yet.
In my case, after adding the 64-bit compiler, I was still getting this error:

Error: NUTC300F:Internal Compiler Error: Native compilation failed due to out of memory error
C:\Program Files (x86)\Microsoft SDKs\UWPNuGetPackages\microsoft.net.native.compiler\2.2.8-rel-28605-00\tools\Microsoft.NetNative.targets(801,5): error : ILT0005: 'C:\Program Files (x86)\Microsoft SDKs\UWPNuGetPackages\runtime.win10-x64.microsoft.net.native.compiler\2.2.8-rel-28605-00\tools\x64\ilc\Tools64\nutc_driver.exe @"<path>\obj\x64\Release\ilc\intermediate\MDIL\<appname>.rsp"' returned exit code -1073741819

As said above, the issue is that not all the toolchain is 64 bit yet.
This is the current state of each stage of the toolchain, related to its 64-bitness:

Tool chain component Purpose Default Bitness Use64BitCompiler
ILC This is the entrypoint and .NET native toolchain orchestrator. It also contains MCG (generates interop marshaling code) and the reducer (analyzes the roots and decides what parts of the original app code to keep or to eliminate). 32-bit 32-bit
Nutc_driver Native UTC compiler. This is responsible for converting the input IL to native code. 32-bit 64-bit
RHBind The x86/x64/arm32 linker. This takes the Nutc_driver output and links it into the final app binary. 32-bit 32-bit
Link This is the arm64 linker and is the same as the native C++ linker. 32-bit 32-bit

In my case, the virtual machine had many vCPU (64) and this was still causing out of memory. See for example the memory comsumption used in my case:

UWP-native-compilation

Doing some researches in Internet, I have found this comment to a GitHub issue, that suggested me to add the SingleThreadNUTC setting, as here:

<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
  ...
  <UseDotNetNativeToolchain>true</UseDotNetNativeToolchain>
  <Use64BitCompiler>true</Use64BitCompiler>
  <SingleThreadNUTC>true</SingleThreadNUTC>
<PropertyGroup />
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
  ...
  <UseDotNetNativeToolchain>true</UseDotNetNativeToolchain>
  <Use64BitCompiler>true</Use64BitCompiler>
  <SingleThreadNUTC>true</SingleThreadNUTC>
<PropertyGroup />

This will use only one thread in the Nutc_driver, slowing it down, but using less memory.
In my personal case, it solved my issue.

Other useful compiler settings to try

The same GitHub issue of the last comment suggests also other settings to try:

  • ShortcutGenericAnalysis: setting it to true can help stop runaway analysis of generic types and reduce overall generation requirements;
  • UseDotNetNativeSharedAssemblyFrameworkPackage: setting it to false eliminates one of the linking boundaries the compiler has to fight with.

Runtime Directives

Another way there it should be possible to optimize memory consumption is the Properties\Default.rd.xml file.
This file is described in Runtime Directives (rd.xml) Configuration File Reference and Runtime Directive Policy Settings; another overview is in .NET Native Deep Dive: Dynamic Features in Static Code.

Essentially, this file contains metadata info for the types defined in your application (including used libraries, like NuGet packages for example) and it's useful for runtime reflection.
Not having this information can lead to the following runtime exceptions, when trying to use or instantiate types that are not included in the .NET Framework: MissingMetadataException or TypeLoadException or NullReferenceException.

The default settings adds all the metadata information for your application, but the size of this information can become massive.

A possible way to optimize/reduce the size of this information is:

  • remove the special Application directive from Properties\Default.rd.xml;
  • get a list of the full set of dlls for your project, for example by inspecting in obj\[architecture]\Release\ilc\in;
  • for each dll, add a Dynamic directive. They’ll look like:
    <Assembly Name="ASSEMBLYNAMEWITHOUTEXTENTION" Dynamic="Required All" />
  • comment out some subset of these libraries;
  • build the app;
  • if the build fails, comment more libraries and retry;
  • if the build doesn't file, test you app and see if you hit any runtime errors;
    • if you do, you'll need to look at the error location and see if adding some directives can help then head back to success runtime test;
    • if you don't, you are fine!