Heap allocations outside malloc/realloc?

[sebjf]

Hi,

I am trying to profile the memory usage of a Unity application. To do this I hook _malloc, _realloc & _free in the wasm, to keep a running total of the allocated chunk sizes in a global which can be read via JS.

When I run our application though, the heap size grows well (100's of Mb) in advance of the tracked allocations.

(To be clear, Unity uses managed code which I believe is transcoded to C++ from CLR bytecode, and then given to Emscripten)

My question is, is there any mechanism to grow or allocate from the heap outside of the exported _malloc & _realloc functions that is known in Emscripten?

(e.g. a non-obvious internal function, or maybe some way for the CLR GC to manage the heap alongside dlmalloc that Unity's toolchain is using?)

[sbc100]

All memory allocations need to go through the single allocator since there is only one single contiguous heap region.

How are you measuring this memory growth? Are you building with Are you talking about the actually wasm memory growing? i.e. are building with ALLOW_MEMORY_GROWTH and seeing the actual Wasm memory object growing?

Or are you talking about memory usage withing the heap managed by the allocator (i.e. dlmalloc?)

[kripken]

Note that there is a trace API that can help with this:

https://emscripten.org/docs/api_reference/trace.h.html

Otherwise, you can manually hook malloc etc., but you'd need to do so for all allocation functions that dlmalloc provides (there is also aligned_alloc for example - see dlmalloc.c). And you need to do so inside the wasm, and not on the JS side.

[sebjf]

Thanks for the replies.

How are you measuring this memory growth? are you talking about memory usage withing the heap managed by the allocator (i.e. dlmalloc?)

So far I have been hooking malloc, realloc & free directly in the wasm to track memory usage within the heap.

For example, here is the _malloc hook (it gets the chunk size of each newly allocated block, masks the status bits and adds it to an accumulator).

(func $_malloc (type 2) (param i32) (result i32)
    (local i32)
    local.get 0
    call $_malloc2
    local.set 1
    local.get 1
    i32.const -4
    i32.add
    i32.load
    i32.const -8 
    i32.and 
    global.get 15
    i32.add
    i32.const 8 
    i32.add 
    global.set 15
    local.get 1)
  (func $_malloc2 (type 2) (param i32) (result i32)
    (local i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32)
    block  ;; label = @1
      block  ;; label = @2

The aim is to accurately track memory consumption to optimise our application that currently runs out of memory unpredictably.

(Though I wasn't able to find dlrealloc_in_place, which is traced with trace.h, in the .wat in order to hook it.)

Note that there is a trace API that can help with this:

Thank you! I have made this work in our Unity application by using the PlayerSettings.WebGL.emscriptenArgs member to provide the --tracing flag, and importing the tracing functions so they can be called through PInvoke in a (managed/C#) Unity Component.

This is an example capture,

From what I understand, HEAP32[DYNAMICTOP_PTR >> 2] points to the top of the chunks (HEAP32[DYNAMICTOP_PTR >> 2] - DYNAMIC_BASE matches the Dynamic metric.), Available is the Buffer size subtracting the chunk top pointer, and Total is the underlying Buffer size?

The Allocation Summary figures seem very low compared to the Memory Usage figures. The Allocation figures match the hooks I place in the WASM. Fragmentation is about 82%. There is a discrepancy of about 600 Mb.

In these tests System.GC.GetTotalMemory() reports about ~500 MiB. I don't really know what to make of this, given what Unity is doing with il2cpp, but it is odd that the size fits so neatly between the reported heap size and the apparent dlmalloc allocations.

Looking at the WAT, the only places $_sbrk is called are $_malloc and $_GC_unix_sbrk_get_mem. If I place an abort in $_GC_unix_sbrk_get_mem I see the trace on start-up showing it called as part of the GC initialisation.

(http://192.168.0.30:8189/unity/Build/unity.framework.js:801:2) at _GC_unix_sbrk_get_mem
(http://192.168.0.30:8189/unity/Build/unity.wasm:wasm-function[58830]:0x116de7e) at _GC_unix_get_mem 
(http://192.168.0.30:8189/unity/Build/unity.wasm:wasm-function[58829]:0x116de75) at _GC_scratch_alloc 
(http://192.168.0.30:8189/unity/Build/unity.wasm:wasm-function[58827]:0x116ddbd) at _GC_init_headers 
(http://192.168.0.30:8189/unity/Build/unity.wasm:wasm-function[58974]:0x11744e1) at _GC_init 
(http://192.168.0.30:8189/unity/Build/unity.wasm:wasm-function[58841]:0x116e5c2) at __ZN6il2cpp2gc16GarbageCollector10InitializeEv 
[il2cpp::gc::GarbageCollector::Initialize()] (http://192.168.0.30:8189/unity/Build/unity.wasm:wasm-function[59487]:0x1185c75) at dynCall_v 
(http://192.168.0.30:8189/unity/Build/unity.wasm:wasm-function[60584]:0x11c2ff6) at Object.dynCall_v 
(http://192.168.0.30:8189/unity/Build/unity.framework.js:25802:36) at invoke_v 
(http://192.168.0.30:8189/unity/Build/unity.framework.js:16873:22) at __ZN6il2cpp2vm7Runtime4InitEPKc

Here is that function,

(func $_GC_unix_sbrk_get_mem (type 2) (param i32) (result i32)
    (local i32 i32)
    i32.const 0
    call $_sbrk
    local.set 1
    i32.const 3040716
    i32.load
    local.tee 2
    i32.const -1
    i32.add
    local.get 1
    i32.and
    local.set 1
    block  ;; label = @1
      local.get 0
      i32.const 0
      i32.lt_s
      if  ;; label = @2
        i32.const 0
        local.set 0
      else
        local.get 1
        if  ;; label = @3
          local.get 2
          local.get 1
          i32.sub
          call $_sbrk
          i32.const -1
          i32.eq
          if  ;; label = @4
            i32.const 0
            local.set 0
            br 3 (;@1;)
          end
        end
        local.get 0
        call $_sbrk
        local.tee 0
        i32.const -1
        i32.eq
        if  ;; label = @3
          i32.const 0
          local.set 0
        end
      end
    end
    local.get 0)

I need to continue to look into it but it looks like the GC might be interacting with the chunk linked list, bypassing the dlmalloc implementation?

(I would guess interacting with the list, rather than just allocating and doing its own thing, since the first call just gets the current location, and surely trying to grab blocks of memory inside the heap throughout the program lifetime would break dlmalloc, since it (dlmalloc) will assume the chunk list is in a contiguous block? I am at the limit now of my knowledge of $_sbrk and how it interacts with DYNAMIC_TOP/DYNAMIC_BASE for now.)

[sbc100]

Some questions:

1. Are you building with ALLOW_MEMORY_GROWTH?
2. What version of emscripten are you using? DYNAMIC_TOP/DYNAMIC_BASE were removed back in 2.0.3 (09/10/2020)

In emscripten, there is only a single sbrk pointer and only one allocator should ever use it. If you are building a GC that uses sbkr() I don't think you can use dlmalloc in the same program since since dlmalloc assumes that it owns all of the memory between __heap_base and the sbrk() pointer. However I would expect that to result in massive memory corruption so I'm surprised your program is working at all.

Here are some useful APIs:

1. emscripten_get_heap_size - returns the current size of the wasm memory (ignore the name heap here.. this is the entire memory, which will never change unless you build with ALLOW_MEMORY_GROWTH).
2. sbkr() returns current end of the heap region that dlmalloc is using to do its allocations. This will always be somewhere between __heap_base and emscripten_get_heap_size().
3. struct mallinfo mallinfo(void); defined in malloc.h.. with this you can ask dlmalloc about its current state.

[sebjf]

Hi,

1. We are building with ALLOW_MEMORY_GROWTH, and the memory does grow.
2. The Unity (2020.3.4f) toolchain uses Emscripten 1.38.11.

I've found the implementation of GC_unix_sbrk_get_mem. (Unity seems to be using BoehmGC)

STATIC ptr_t GC_unix_sbrk_get_mem(size_t bytes)
{
  ptr_t result;
# ifdef IRIX5
    /* Bare sbrk isn't thread safe.  Play by malloc rules.      */
    /* The equivalent may be needed on other systems as well.   */
    __LOCK_MALLOC();
# endif
  {
    ptr_t cur_brk = (ptr_t)sbrk(0);
    SBRK_ARG_T lsbs = (word)cur_brk & (GC_page_size-1);

    if ((SBRK_ARG_T)bytes < 0) {
        result = 0; /* too big */
        goto out;
    }
    if (lsbs != 0) {
        if((ptr_t)sbrk((SBRK_ARG_T)GC_page_size - lsbs) == (ptr_t)(-1)) {
            result = 0;
            goto out;
        }
    }
#   ifdef ADD_HEAP_GUARD_PAGES
      /* This is useful for catching severe memory overwrite problems that */
      /* span heap sections.  It shouldn't otherwise be turned on.         */
      {
        ptr_t guard = (ptr_t)sbrk((SBRK_ARG_T)GC_page_size);
        if (mprotect(guard, GC_page_size, PROT_NONE) != 0)
            ABORT("ADD_HEAP_GUARD_PAGES: mprotect failed");
      }
#   endif /* ADD_HEAP_GUARD_PAGES */
    result = (ptr_t)sbrk((SBRK_ARG_T)bytes);
    if (result == (ptr_t)(-1)) result = 0;
  }
 out:
# ifdef IRIX5
    __UNLOCK_MALLOC();
# endif
  return(result);
}

It looks to me as if it is trying to get more memory from process via sbrk, aligned to the gc page size.

It is not just at start-up either, but seems to be called while dlmalloc is in use too. Here is an example stack trace for one invocation, well after the application has started.

$_GC_unix_sbrk_get_mem (unity.wasm:0x55fe1)
$_GC_unix_get_mem (unity.wasm:0x116ded3)
$_GC_expand_hp_inner (unity.wasm:0x116f29b)
$_GC_collect_or_expand (unity.wasm:0x116ef4c)
$_GC_allocobj (unity.wasm:0x116eace)
$_GC_generic_malloc_inner (unity.wasm:0x116e8a7)
$_GC_generic_malloc (unity.wasm:0x1174d70)
$_GC_malloc_kind (unity.wasm:0x1174ccf)
$_GC_malloc_atomic (unity.wasm:0x1174da0)
$__ZN6il2cpp2vm6Object15AllocatePtrFreeEmP11Il2CppClass (unity.wasm:0x1141eb4)
$__ZN6il2cpp2vm6String7NewSizeEi (unity.wasm:0x114245d)

From what I can tell, dlmalloc seems to be able to handle other parts of the program calling sbrk. I don't understand the algorithm exactly, but comments like this and this suggest dlmalloc is OK with operating on a non-contiguous heap. I would guess that BoehmGC is similarly tolerant.

That seems to solve the confusion then: there is another allocator, just not known to Emscripten because Unity put it there.

(I can imagine this is a nightmare for fragmentation however!)

Thank you for all the pointers!