Memory

Table of Contents

• Features
  • Classes
• Status
• Description
  • Buffer
  • Memory
  • VirtualMemory
  • Globals
• Blog
• Roadmap

🟩 Heap Allocation, Custom allocators, Virtual Memory, Buffer, Segment

Memory library helps tracking and limiting runtime / dynamic allocations through the use of custom allocators.

If a library doesn't directly or indirectly depend on the Memory library, you can assume that it will not do any runtime / dynamic allocation.

Most classes have been originally part of the Foundation library.

Features

Classes

Class	Description
SC::Buffer	An heap allocated byte buffer that can optionally use an inline buffer.
SC::Memory	Centralized functions to allocate, reallocate and deallocate memory.
SC::VirtualMemory	Reserves a contiguous slice of virtual memory committing just a portion of it.
SC::Globals	Customizable thread-local and global variables for memory handling.

Status

🟩 Usable
The library is solid. The Buffer implementation has been evolved and fine tuned to be minimal but effective.

Description

Memory library helps tracking and limit runtime / dynamic allocations through the use of custom allocators. A classic dynamically expandable binary buffer SC::Buffer is provided and it's largely shared to form the more object model oriented SC::Vector class from Containers Library.

All allocations throughout all downstream dependant libraries are centrally tracked by the SC::Globals class, that also allows re-defining custom thread-local allocators.

Such allocators can be just fixed buffers, regular heap memory or reserved SC::VirtualMemory using only limited amounts of Physical memory.

Buffer

An heap allocated byte buffer that can optionally use an inline buffer.

See also

SC::SmallBuffer to use an inline buffer that can optionally become heap allocated as needed.

Note

This class (and SC::SmallBuffer) reduces needs for the header-only SC::Vector (from Containers). SC::Buffer avoids some compile time / executable size bloat because it's not header only.

Example:

bool funcRequiringBuffer(Buffer& buffer)
{
    for (size_t idx = 0; idx < buffer.size(); ++idx)
    {
        if (buffer[idx] != 123)
            return false;
    }
    return true;
}
 
void BufferTest::basic()
{
    Buffer buffer;
    // Allocate 16 bytes
    SC_TEST_EXPECT(buffer.resizeWithoutInitializing(16));
 
    // Buffer is not inline (it's heap allocated)
    SC_TEST_EXPECT(not buffer.isInline());
 
    // Fill buffer with a value
    buffer.clear();
    SC_TEST_EXPECT(buffer.resize(buffer.capacity(), 123));
    funcRequiringBuffer(buffer);
 
    // Declare a buffer with inline capacity of 128 bytes
    SmallBuffer<128> smallBuffer;
 
    // copy buffer (will not allocate)
    smallBuffer = buffer;
 
    // smallBuffer is using inline buffer (no heap allocation)
    SC_TEST_EXPECT(smallBuffer.isInline());
    SC_TEST_EXPECT(smallBuffer.size() == 16);
    SC_TEST_EXPECT(smallBuffer.capacity() == 128);
 
    // SmallBuffer can be passed in place of regular Buffer
    funcRequiringBuffer(smallBuffer);
 
    SC_TEST_EXPECT(buffer.resizeWithoutInitializing(1024));
 
    // SmallBuffer now will allocate 1024 bytes
    // by using assignCopy instead of assignment operator
    // caller can check for allocation failure
    SC_TEST_EXPECT(smallBuffer.assign(buffer.toSpanConst()));
    SC_TEST_EXPECT(not smallBuffer.isInline());
    SC_TEST_EXPECT(smallBuffer.size() == 1024);
    SC_TEST_EXPECT(smallBuffer.capacity() == 1024);
 
    // Allocate 2kb on another buffer
    Buffer buffer2;
    SC_TEST_EXPECT(buffer2.resizeWithoutInitializing(2048));
 
    // SmallBuffer will "steal" the 2Kb buffer
    smallBuffer = move(buffer2);
 
    SC_TEST_EXPECT(smallBuffer.size() == 2048);
    SC_TEST_EXPECT(smallBuffer.capacity() == 2048);
    SC_TEST_EXPECT(buffer2.isEmpty());
 
    // Resize small buffer to its original capacity
    SC_TEST_EXPECT(smallBuffer.resizeWithoutInitializing(128));
 
    // The heap block is still in use
    SC_TEST_EXPECT(not smallBuffer.isInline());
    SC_TEST_EXPECT(smallBuffer.capacity() == 2048);
 
    // Shrinking it will restore its original inline buffer
    SC_TEST_EXPECT(smallBuffer.shrink_to_fit());
 
    // And verify that that's actually true
    SC_TEST_EXPECT(smallBuffer.isInline());
    SC_TEST_EXPECT(smallBuffer.capacity() == 128);
}

Memory

Centralized functions to allocate, reallocate and deallocate memory.

VirtualMemory

Reserves a contiguous slice of virtual memory committing just a portion of it.

This class is useful on 64-bit systems where the address space is so large that it's feasible reserving large chunks of memory to commit and de-commit (shrink) as needed.
Reservation ensures that the returned address will not change and will be sized in multiples of system page size.

Note

Memory must be committed in order to be read or written, occupying physical memory pages.

Warning

This class has no defined destructor so memory MUST be released calling VirtualMemory::release

    // This test uses two pages initially and just one page later
    // On Windows and Linux default Page size is typically 4Kb
    // On macOS default page size is typically 16 Kb
    const size_t moreThanOnePageSize = VirtualMemory::getPageSize() + 1024;
    const size_t lessThanOnePageSize = VirtualMemory::getPageSize() - 1024;
    SC_TEST_EXPECT(lessThanOnePageSize > 0); // sanity check just in case
 
    void* reference = Memory::allocate(moreThanOnePageSize, 1);
    memset(reference, 1, moreThanOnePageSize);
    auto releaseLater = MakeDeferred([&] { Memory::release(reference); });
 
    VirtualMemory virtualMemory;
 
    // Reserve 2 pages of virtual memory
    SC_TEST_EXPECT(virtualMemory.reserve(2 * VirtualMemory::getPageSize()));
 
    // Request to use less than one page of virtual memory
    SC_TEST_EXPECT(virtualMemory.commit(lessThanOnePageSize));
    char* memory = static_cast<char*>(virtualMemory.memory);
 
    // Check that memory is writable and fill it with 1
    memset(memory, 1, lessThanOnePageSize);
 
    // Let's now extend this block from one to two pages
    SC_TEST_EXPECT(virtualMemory.commit(moreThanOnePageSize));
 
    // Fill the "newly committed" pages with 1
    memset(memory + lessThanOnePageSize, 1, moreThanOnePageSize - lessThanOnePageSize);
 
    // Make sure that previously reserved address is stable
    SC_TEST_EXPECT(memory == virtualMemory.memory);
 
    // Check that all allocated bytes are addressable and contain expected pattern
    SC_TEST_EXPECT(memcmp(memory, reference, moreThanOnePageSize) == 0);
 
    // Now let's de-commit everything but the first page
    SC_TEST_EXPECT(virtualMemory.shrink(lessThanOnePageSize));
 
    // Address should stay stable
    SC_TEST_EXPECT(memory == virtualMemory.memory);
    SC_TEST_EXPECT(memcmp(memory, reference, lessThanOnePageSize) == 0);
 
    // Decommit everything (not really needed if we're going to release() soon)
    SC_TEST_EXPECT(virtualMemory.shrink(0));
    SC_TEST_EXPECT(memory == virtualMemory.memory);
 
    // Finally release (don't forget, VirtualMemory has no destructor!)
    SC_TEST_EXPECT(virtualMemory.release());
    SC_TEST_EXPECT(virtualMemory.memory == nullptr);

Globals

Customizable thread-local and global variables for memory handling.

This class holds pointers to systems that must be globally available, like the memory allocator. It allows "stacking" different Globals through a push / pop mechanism, connecting them through a linked list. The Default allocator is automatically setup and uses standard malloc, realloc, free for allocations.

Note

Globals use no locking mechanism so they are thread-unsafe. Every method however requires a Globals::Type parameter that can be set to Globals::ThreadLocal to avoid such issues.

Example (Fixed Allocator):

    alignas(uint64_t) char stackMemory[256] = {0};
 
    FixedAllocator fixedAllocator = {stackMemory, sizeof(stackMemory)};
    Globals        globals        = {fixedAllocator};
    Globals::push(Globals::Global, globals);
    // ...
    Buffer& buffer = *Globals::get(Globals::Global).allocator.create<Buffer>();
    (void)buffer.append({"ASDF"}); // Allocates from stackMemory
    // ...
    Globals::pop(Globals::Global);

Example (Virtual Allocator):

    // Create a Virtual memory block that can expand up to 1 MB
    VirtualMemory virtualMemory;
    SC_TEST_EXPECT(virtualMemory.reserve(1024 * 1024));
 
    VirtualAllocator virtualAllocator = {virtualMemory};
    Globals          virtualGlobals   = {virtualAllocator};
    Globals::push(Globals::Global, virtualGlobals);
    // ...
    Buffer& buffer = *Globals::get(Globals::Global).allocator.create<Buffer>();
    (void)buffer.append({"ASDF"}); // Allocates from virtualMemory
    // ...
    Globals::pop(Globals::Global);

Example (Memory dump):

    // -----------------------------------------------------------------------------
    // Example showing how to dump and restore a complex struct to a flat buffer.
    // SC::Segment based containers use relative pointers to make this possible.
    // DO NOT use this approach when versioning is needed, that means needing to
    // de-serialize after adding, removing or moving fields in the structure.
    // In such cases consider using SC::SerializationBinary (versioned reflection).
    // -----------------------------------------------------------------------------
    struct NestedStruct
    {
        VectorMap<String, int> someMap;
        VectorSet<int>         someSet;
    };
    struct ComplexStruct
    {
        Vector<String> someStrings;
        int            someField = 0;
        String         singleString;
        NestedStruct   nestedStruct;
    };
 
    Buffer memoryDump;
 
    // Setup a Virtual Memory allocator with the max upper memory bound
    VirtualMemory virtualMemory;
    SC_TEST_EXPECT(virtualMemory.reserve(1024 * 1024)); // 1MB is enough here
    VirtualAllocator allocator = {virtualMemory};
    Globals          globals   = {allocator};
 
    // Make the allocator current before creating a ComplexStruct
    Globals::push(Globals::Global, globals);
    ComplexStruct& object = *allocator.create<ComplexStruct>();
    object.someField      = 42;
    object.singleString   = "ASDF";
    object.someStrings    = {"First", "Second"};
    SC_TEST_EXPECT(object.nestedStruct.someSet.insert(213));
    SC_TEST_EXPECT(object.nestedStruct.someMap.insertIfNotExists({"1", 1}));
 
    // Save used bytes to memoryDump, checking that one page has been committed
    Span<const void> memory = {allocator.data(), allocator.size()};
    SC_TEST_EXPECT(virtualMemory.committedBytes == virtualMemory.getPageSize());
    SC_TEST_EXPECT(memory.sizeInBytes() < virtualMemory.getPageSize());
    SC_TEST_EXPECT(memory.data() == &object);
    SC_TEST_EXPECT((size_t(memoryDump.data()) % alignof(ComplexStruct)) == 0);
    Globals::pop(Globals::Global);
 
    // Dump AFTER Globals::pop, using default allocator, and release virtual memory
    SC_TEST_EXPECT(memoryDump.append(memory));
    SC_TEST_EXPECT(virtualMemory.release());
 
    // -----------------------------------------------------------------------------
    // Obtain a read-only view over ComplexStruct by re-interpreting the memory dump
    // NOTE: There's no need to call ComplexStruct destructor at end of scope
    // WARN: start_lifetime_as obtains a ComplexStruct with proper lifetime.
    // It works on all tested compilers (debug and release) but it's not technically
    // UB-free as ComplexStruct is not implicit-lifetime.
    // -----------------------------------------------------------------------------
    const Span<const void> span     = memoryDump.toSpanConst();
    const ComplexStruct&   readonly = *span.start_lifetime_as<const ComplexStruct>();
    SC_TEST_EXPECT(readonly.someField == 42);
    SC_TEST_EXPECT(readonly.singleString == "ASDF");
    SC_TEST_EXPECT(readonly.someStrings[0] == "First");
    SC_TEST_EXPECT(readonly.someStrings[1] == "Second");
    SC_TEST_EXPECT(readonly.someStrings.size() == 2);
    SC_TEST_EXPECT(readonly.nestedStruct.someSet.size() == 1);
    SC_TEST_EXPECT(readonly.nestedStruct.someSet.contains(213));
    SC_TEST_EXPECT(*readonly.nestedStruct.someMap.get("1") == 1);
 
    // -----------------------------------------------------------------------------
    // To modify the struct again, copy the read-only view to a new object.
    // A Fixed or Virtual allocator can be used here to group sparse allocations in
    // a nice single contiguous buffer, before dumping it again to disk or network.
    // -----------------------------------------------------------------------------
    ComplexStruct modifiable = readonly;
    SC_TEST_EXPECT(modifiable.someStrings[0] == "First");
    modifiable.someStrings[0] = "First modified";
    SC_TEST_EXPECT(modifiable.someStrings[0] == "First modified");

Blog

Some relevant blog posts are:

(These blogs were written before the split from Foundation into the Memory library)

• February 2025 Update
• March 2025 Update
• April 2025 Update

Roadmap

🟦 Complete Features:

• Things will be added as needed

💡 Unplanned Features:

• SharedPtr
• UniquePtr

Note

In Principles there is a rule that discourages allocations of large number of tiny objects and also creating systems with unclear or shared memory ownership. For this reason this library is missing Smart Pointers.