Best way to buffer small objects in memory?

[kitfox]

I need to buffer a bunch of small objects in memory as part of a Variant class I'm creating. Each Variant needs to store a serialized version of the object it is managing. I'm thinking of logic something like the following:

template <typename T>
int8_t* store(const T& foo) {
    int8_t* buf = new int8_t[sizeof(T)];
    memcpy(buf, &foo, sizeof(T));
    return buf;
}

template <typename T>
T create(const int8_t* buf) {
    T value;
    memcpy(&value, buf, sizeof(T));
    return value;
}

I realize that the above is problematic due to raw pointers and raw memory copying, but I'm not sure what a better way to do it would be. The majority of the Variants I use will be tracking very small objects like floats, ints and bools, but they need to be flexible enough to handle larger structs and classes too.

What would be the best data type for my buffer object? I want something fast and which could avoid fragmenting memory if possible.

[helios]

The above code is too trivial to handle classes. What do you do for cases like these?

class Node{
public:
    Node *next;
};

auto node = new Node;
node->next = node;

Or these:

class Base{
public:
    virtual ~Base();
};

class DerivedA : virtual Base{
public:
    virtual ~DerivedA();
};

class DerivedB : virtual Base{
public:
    virtual ~DerivedB();
};

class DerivedC : public DerivedA, public DerivedB{
};

First define very clearly the scope of the problem you're trying to solve, and then attempt to solve it, and finally don't worry if your solution doesn't solve problems outside your defined scope.

[jonnin]

Once you get into it, consider a design via bottom up. You need your low level serialize stuff that can be safely memcpy or file.write() or sendto() etc as byte arrays to be built out first and layer on top of it. If you try to start at a high level or mid level and go down to the low level, you will have more problems with things like stl containers that are a pain to serialize. There is a time and a place for C code, and it sounds like you may want a C bottom level with C++ wrappers over it. But, it also sounds like you may be trying to solve the wrong problem or prematurely optimize or something as well, and that issue ties into the scope of the problem comments above.

At the end of the day, you can code literally anything into byte arrays. Doing so blocks use of some tools that make coding easier, cleaner, safer, faster, etc. You can't use anything dynamic, like containers. You can't have flexible types, like templates (but you can use a union surrogate there for the types you specifically need in your solution). Your problem scope and solution may have you doing something like this, down in the bytes, but before you do that, if it seems like this is where you will land for your design etc, I would post a detailed problem statement here and let people guide you away from such a low level, raw approach.