Description
Description
I'm trying to embed the Mono runtime into a native C++ application.
Everything worked fine under Net8, but after upgrading to Net10 preview4 the program doesn't work properly.
Reproduction Steps
Minimal reproducible program:
namespace AotHelloWorld;
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello AOT world!");
Console.ReadLine(); // Throw System.OutOfMemoryException in Net10 P4
Console.WriteLine("Bye AOT world!");
}
}
System.Private.CoreLib.dll AOT image is build with:
mono-sgen --aot=full,interp,static System.Private.CoreLib.dll
mono runtime is build with:
build.cmd mono+libs -c release -cmakeargs "-DDISABLE_JIT=TRUE -DENABLE_OVERRIDABLE_ALLOCATORS=TRUE"
Expected behavior
The program should run normally and output
Hello AOT world!
Bye AOT world!
Actual behavior
The program output Hello AOT world! then throw an unexpected exception
Unhandled Exception:
System.OutOfMemoryException: Insufficient memory to continue the execution of the program.
at System.IO.StreamReader..ctor(Stream stream, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean leaveOpen)
at System.ConsolePal.GetOrCreateReader()
at System.Console.<get_In>g__EnsureInitialized|14_0()
at System.Console.get_In()
at System.Console.ReadLine()
at AotHelloWorld.Program.Main(String[] args)
Regression?
Works fine in Net8
Known Workarounds
n/a
Configuration
10.0.100-preview.4.25258.110
Windows 11 x64
Other information
System.IO.StreamReader..ctor calls encoding.GetMaxCharCount(bufferSize) to get _maxCharsPerBuffer.
Since System.Console.dll does not have AOT, the interpreter is used here to execute this function.
The problem is that the interpreter does not correctly set the register when passing the return value back to the AOT code.
At the end of interp_entry_from_trampoline in interp.c, mono_arch_set_native_call_context_ret is called to write the return value calculated by the interpreter back to AOT.
For the GetMaxCharCount function, the return value is placed in gregs[AMD64_RAX] of CallContext. However, since the return type is int32, the stackval_to_data function only fills the lower 32 bits of RAX, while the upper 32 bits remain unchanged, resulting in an incorrect value in RAX.
After checking the code of Net8 and the history of mini-amd64.c, I found that mono_arch_set_native_call_context_ret used to have a code memset (ccontext, 0, sizeof (CallContext)); // FIXME to clear the initial value of the register, but this line was removed after 4185f94. I tried adding this line back, and then the test program can run normally, but I'm not sure if this is the right way to fix it.
Also, although I haven't tested it, I suspect arm64 version has similar issues.