Skip to content

System.OutOfMemoryException when allocating jump stub on macOS ARM64 #83818

New issue
New issue
Closed
#84023
#83819
Closed
System.OutOfMemoryException when allocating jump stub on macOS ARM64#83818
#84023
#83819
Labels
area-VM-coreclr
@k15tfu

Description

@k15tfu
k15tfu
opened on Mar 23, 2023

Hi!

Sometimes I get a System.OutOfMemoryException exception in .NET 7 app on macOS ARM64 in arbitrary places like:

System.OutOfMemoryException
at InvokeStub_ContextConsumersWithAttributesManager..ctor(Object, Object, IntPtr*)
at System.Reflection.ConstructorInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

Digging into this, it turned out that the exception is thrown from HostCodeHeap::InitializeHeapList():

runtime/src/coreclr/vm/dynamicmethod.cpp

Lines 442 to 449 in d037e07

pTracker = AllocMemory_NoThrow(0, JUMP_ALLOCATE_SIZE, sizeof(void*), 0);
if (pTracker == NULL)
{
// This should only ever happen with fault injection
_ASSERTE(g_pConfig->ShouldInjectFault(INJECTFAULT_DYNAMICCODEHEAP));
delete pHp;
ThrowOutOfMemory();
}

because ExecutableAllocator::Commit() failed to commit executable memory:

runtime/src/coreclr/vm/dynamicmethod.cpp

Lines 758 to 762 in d037e07

if (NULL == ExecutableAllocator::Instance()->Commit(m_pLastAvailableCommittedAddr, sizeToCommit, true /* isExecutable */))
{
LOG((LF_BCL, LL_ERROR, "CodeHeap [0x%p] - VirtualAlloc failed\n", this));
return NULL;
}

On macOS ARM64 ExecutableAllocator::IsDoubleMappingEnabled() is disabled, thus it internally calls VirtualAlloc(..., MEM_COMMIT, PAGE_EXECUTE_READWRITE) for the pre-reserved memory HostCodeHeap::m_pLastAvailableCommittedAddr, but then mprotect(..., PROT_EXEC | PROT_READ | PROT_WRITE) fails with errno EACCES (aka Permission denied):

runtime/src/coreclr/pal/src/map/virtual.cpp

Lines 1202 to 1216 in d037e07

if (protectionState != vProtect)
{
// Change permissions.
if (mprotect((void *) StartBoundary, MemSize, nProtect) != -1)
{
memset(pInformation->pProtectionState + runStart,
vProtect, runLength);
}
else
{
ERROR("mprotect() failed! Error(%d)=%s\n",
errno, strerror(errno));
goto error;
}
}

because initially this memory region was reserved w/o MEM_RESERVE_EXECUTABLE flag (which is used for MAP_JIT):

runtime/src/coreclr/utilcode/util.cpp

Lines 448 to 460 in d037e07

if ((mbInfo.State == MEM_FREE) &&
(mbInfo.RegionSize >= (SIZE_T) dwSize || mbInfo.RegionSize == 0))
{
// Try reserving the memory using VirtualAlloc now
pResult = (BYTE*)ClrVirtualAlloc(tryAddr, dwSize, MEM_RESERVE, flProtect);
// Normally this will be successful
//
if (pResult != nullptr)
{
// return pResult
break;
}

despite the fact that ExecutableAllocator::ReserveWithinRange() actually requests it:

runtime/src/coreclr/utilcode/executableallocator.cpp

Lines 595 to 605 in d037e07

else
{
DWORD allocationType = MEM_RESERVE;
#ifdef HOST_UNIX
// Tell PAL to use the executable memory allocator to satisfy this request for virtual memory.
// This will allow us to place JIT'ed code close to the coreclr library
// and thus improve performance by avoiding jump stubs in managed code.
allocationType |= MEM_RESERVE_EXECUTABLE;
#endif
return ClrVirtualAllocWithinRange((const BYTE*)loAddress, (const BYTE*)hiAddress, size, allocationType, PAGE_NOACCESS);
}

Linked issues: #50391, #54954.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-VM-coreclr

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions