unique complie time variable creation for templates

[flogakid]

Sorry about the confusing subject line, but it's a kind of complex question.
What I'm trying to do is have a templated member function of a class in which each different generated function has access to a variable unique to that instantiation of the function alone. It is important that this variable is somehow determined at compile time, as there is no guarantee as to which order the functions will be called, and all the variables must be the same every execution. I have attempted to think my way through this problem many times and I'm simply at a loss. any help would be greatly appreciated.
also, sorry if I wasn't clear enough, please feel free to ask for clarification on any given point.

[moorecm]

I think an example attempt might better describe your goal.

Do you mean like a static variable inside the function template? For example, you could accumulate the number of times the function was called (unique to that instantiation) or something similar. I'm not clear as to how this would be very useful, so this is probably not what you're after...

[flogakid]

Ok, let me try again. I'm attempting to make a networked program that can send various types of messages across a network. To accomplish this I basically want to have a templatized "add to message" function that you can simply give a type and then the data, and it will add it to the message. The problem is that the computer on the other end has to know what type I'm sending, there for every unique (Sorry if i don't know the correct wording) version of the templated function (i.e. one for floats, one for doubles, ect...). and for that i need each version to get it's own unique ID that it can use, and since i can't guarantee execution order of the functions, the ID's must be generated at compile time (or some other means that guarantee the same ID's are created for each instance of the program). I know I could do this with an enum or something that you just pop the type into, but i would really like to have it such that all you have to do is use the function, and it works. I know that C++ generates the code for each version of the member function at compile time. so the different versions exist, i just need each one to get a different number that it can use.
hopefully that clarify things.

[Incubbus]

Let me try to say it in my words (for clarification and that I did understand your problem):

You want the client side to know which function to call without passing information about the message data sent?...

Is this even possible?...
I would suggest You the insertion of some kind of header into your message... so your client application can find out which function to call (using function pointers e.g.)...

<- Or is this the same as You mentioned with the enumeration stuff?....

[flogakid]

No, I'll be sending messages, but the idea is that I will be sending messages that could contain many different types of things. Therefore, each given type of thing i can add to a message needs to have it's own unique id, so that the receiver knows how to interpret the data. but i don't want to have to add many, many functions for each different data type (or for that matter structs or so on) so i would make the function templatized. only problem is now they don't have a unique identifier associated with the type I'm adding. so i need to somehow make one, but i cant make them at run time as they must be the same throughout all of the programs, and i can't guarantee the order in which the functions are first called.
let me try to give a psudocode example of how it would be used:
int a;
float b;
double c;
randomstruct d;
Message message;
message.add<int>(a);
message.add<float>(b);
message.add<double>(c);
message.add<randomstruct>(d);
message.send();

where add would be the tempolated member function.
all of these values would be essentially stored in a character buffer, with no way of telling what they were. that is, except for the ID it would also add to a buffer elsewhere. but for that to work those adds must add unique IDs based on the template type. these ID must be the same for every instance running, and i can't guarantee execution order. so the ID would need to be generated somehow at compile time(i think). hopefully this clarifies things more :P

[moorecm]

I like how your thinking about this. The problem, however, is that I don't see how that can be generalized. For example, even if you could embed a what-type-follows enum or even a header of what the message contains, how would you store it at the other end? Wouldn't the format of the message still have to be known at both sides?

The messaging implementations that I am familiar with have a similar scheme of using overridden "add to message" and "get from message" methods/functions but there is still a header shared among both sides (sender and receiver) that defines the message format.

[flogakid]

yes, both sides would know what the message looked like, it would just need to receive a unique ID so that it could interpret it as such. the server and client are one in the same, there are simply going to be multiple copies of it running. So everyone will know these types. the only problem I'm having is generating those unique id's.

[moorecm]

The only thing that comes to mind is what you already mentioned before about putting an enumeration in the shared header. Maybe someone else will have an idea...

[kbw]

I can't see how you can automate the generation of an Id in a deterministic way other than building a hash from the name of the type. But then, your hash would need to be large enough to aviod clashes and you'd have sparse use of the number, leading you to pass more bytes than strictly necessary for the type id.

I'm not even sure you can pass anything. What if one of your anythings was a list of vector of some struct? How would you automatically decode that on the receiving end?

[m4ster r0shi]

I think you can do something like this:

const string id_table[]={"int","long","float","double","my_imba_struct" /*,...*/};

class MsgData {public: virtual string id()=0; /*...*/}; //abstract class

class MsgInt:public MsgData {private:int value;
public:string id() {return id_table[0];} /*...*/};

class MsgLong:public MsgData {private:long value;
public:string id() {return id_table[1];} /*...*/};

class MsgFloat:public MsgData {private:float value;
public:string id() {return id_table[2];} /*...*/};

class MsgDouble:public MsgData {private:double value;
public:string id() {return id_table[3];} /*...*/};

class MsgMyImbaStruct:public MsgData {private: my_imba_struct * value;
public:string id() {return id_table[4];} /*...*/}

Then implement your Message::add function like this:

Message::add(const MsgData & data)
{
     //...
     data_type=data.id();
     actual_data=data.value;
    //...
}

As you can see, you can do this without having to use templates!

[m4ster r0shi]

Ok, I've got something I think you'll like :) The following example demonstrates conversion of data from a bunch of ints, doubles and strings to a byte sequence (vector of chars) and vice versa. Enjoy!

#include <iostream>
#include <string>
#include <vector>
using namespace std;

//declarations...

class Message;

//abstract Message Data class
class MsgData
{
    public:
    virtual bool input_value(){return true;}
    virtual void output_value() const{}
    virtual int get_id() const{return -1;}
    virtual int get_size() const{return 0;}
    virtual void add_to_message(Message * msg) const{}
    virtual void read_from_message(Message * msg, int start, int & stop){}
};

//Integer Message Data class
class MsgInt:public MsgData
{
    protected:
    int value;

    public:
    bool input_value();
    void output_value() const;

    int get_id() const;
    int get_size() const;

    void add_to_message(Message * msg) const;
    void read_from_message(Message * msg, int start, int & stop);
};

//Double Message Data class
class MsgDouble:public MsgData
{
    protected:
    double value;

    public:
    bool input_value();
    void output_value() const;

    int get_id() const;
    int get_size() const;

    void add_to_message(Message * msg) const;
    void read_from_message(Message * msg, int start, int & stop);
};

//String Message Data class
class MsgString:public MsgData
{
    protected:
    string value;

    public:
    bool input_value();
    void output_value() const;

    int get_id() const;
    int get_size() const;

    void add_to_message(Message * msg) const;
    void read_from_message(Message * msg, int start, int & stop);
};

//Message class
class Message
{
    friend void MsgData::add_to_message(Message * msg) const;
    friend void MsgData::read_from_message(Message * msg, int start, int & stop);
    friend void MsgInt::add_to_message(Message * msg) const;
    friend void MsgInt::read_from_message(Message * msg, int start, int & stop);
    friend void MsgDouble::add_to_message(Message * msg) const;
    friend void MsgDouble::read_from_message(Message * msg, int start, int & stop);
    friend void MsgString::add_to_message(Message * msg) const;
    friend void MsgString::read_from_message(Message * msg, int start, int & stop);

    vector<char> data;

    public:
    void add(const MsgData * msg_data);
    void show();
};

//********************************************************************************
//(id->message data class)-mapping
MsgData * NewMsgInt() {MsgData * msg_data=new MsgInt; return msg_data;}
MsgData * NewMsgDouble() {MsgData * msg_data=new MsgDouble; return msg_data;}
MsgData * NewMsgString() {MsgData * msg_data=new MsgString; return msg_data;}

typedef MsgData * (*NewMsgDataFunc)(void);
const NewMsgDataFunc new_msg_data_ftable[]={NewMsgInt,NewMsgDouble,NewMsgString};

MsgData * NewMsgData(int id)
{
    MsgData * msg_data=new_msg_data_ftable[id]();
    return msg_data;
}
//********************************************************************************

//definitions...

bool MsgInt::input_value()
{
    cout << "enter an int: ";

    bool success=(cin>>value);
    cin.clear();

    while (cin.get()!='\n');

    return success;
}

void MsgInt::output_value() const
{
    cout << "your int is: ";
    cout << value;
}

int MsgInt::get_id() const {return 0;}

int MsgInt::get_size() const {return sizeof(int);}

void MsgInt::add_to_message(Message * msg) const
{
    char data_id=get_id();
    char data_size=get_size();

    msg->data.push_back(data_id);
    msg->data.push_back(data_size);

    int i;
    char * cur_byte=(char*)&value;
    for (i=0; i<data_size; i++)
    {
        msg->data.push_back(*cur_byte);
        cur_byte++;
    }
}

void MsgInt::read_from_message(Message * msg, int start, int & stop)
{
    int end=get_size()+start+2;

    int i;
    char cur_byte;

    value=0;
    for (i=start+2; i<end; i++)
    {
        cur_byte=msg->data[i];
        *(((char*)&value)+i-start-2)=cur_byte;
    }

    stop=end;
}

bool MsgDouble::input_value()
{
    cout << "enter a double: ";

    bool success=cin>>value;
    cin.clear();
    while (cin.get()!='\n');

    return success;
}

void MsgDouble::output_value() const
{
    cout << "your double is: ";
    cout << value;
}

int MsgDouble::get_id() const {return 1;}

int MsgDouble::get_size() const {return sizeof(double);}

void MsgDouble::add_to_message(Message * msg) const
{
    char data_id=get_id();
    char data_size=get_size();

    msg->data.push_back(data_id);
    msg->data.push_back(data_size);

    int i;
    char * cur_byte=(char*)&value;
    for (i=0; i<data_size; i++)
    {
        msg->data.push_back(*cur_byte);
        cur_byte++;
    }
}

void MsgDouble::read_from_message(Message * msg, int start, int & stop)
{
    int end=get_size()+start+2;

    int i;
    char cur_byte;

    value=0;
    for (i=start+2; i<end; i++)
    {
        cur_byte=msg->data[i];
        *(((char*)&value)+i-start-2)=cur_byte;
    }

    stop=end;
}

bool MsgString::input_value()
{
    cout << "enter a string (at most 100 chars): ";
    getline(cin,value,'\n');

    bool success=cin;
    cin.clear();

	if (value.size()>100) success=false;

    return success;
}

void MsgString::output_value() const
{
    cout << "your string is: ";
    cout << value;
}

int MsgString::get_id() const {return 2;}

int MsgString::get_size() const {return value.size();}

void MsgString::add_to_message(Message * msg) const
{
    char data_id=get_id();
    char data_size=get_size();

    msg->data.push_back(data_id);
    msg->data.push_back(data_size);

    int i;
    char cur_byte;
    for (i=0; i<data_size; i++)
    {
        cur_byte=value[i];
        msg->data.push_back(cur_byte);
    }
}

void MsgString::read_from_message(Message * msg, int start, int & stop)
{
    int end=msg->data[start+1]+start+2;

    int i;
    char cur_byte;

    value.clear();
    for (i=start+2; i<end; i++)
    {
        cur_byte=msg->data[i];
        value+=cur_byte;
    }

    stop=end;
}

void Message::add(const MsgData * msg_data) {msg_data->add_to_message(this);}

void Message::show()
{
    system("cls");

    int start=0;
    int stop;
    int end=data.size();

    MsgData * cur_msg_data;
    int id;

    if (end==0) {cout << "your message is empty!" << endl; system("pause"); return;}

    while (true)
    {
        id=data[start];

        cur_msg_data=NewMsgData(id);

        cur_msg_data->read_from_message(this,start,stop);

        cur_msg_data->output_value();
        cout << endl;

        delete cur_msg_data;

        start=stop;

        if (start>=end) break;
    }
    system("pause");
}


void show_menu()
{
    system("cls");
    cout << "(1) add an int\n";
    cout << "(2) add a double\n";
    cout << "(3) add a string\n";
    cout << "(4) show message\n";
    cout << "(5) quit..." << endl;
}

int main()
{
    Message message;
    MsgData * cur_msg_data;

    int choice;

    while (true)
    {
        show_menu();
        cin >> choice;
        cin.clear();
        while (cin.get()!='\n');

        if (choice<1||choice>5) continue;

        if (choice==4) {message.show(); continue;}
        if (choice==5) break;

        cur_msg_data=NewMsgData(choice-1);

        while(!cur_msg_data->input_value());

        message.add(cur_msg_data);
        delete cur_msg_data;
    }

    system("cls");
    cout << "bye!" << endl;
    system("pause");

    return 0;
}

[Galik]

Are you looking for something like this, using RTTI?

#include <typeinfo>
#include <iostream>
#include <fstream>
#include <string>

class connection_t
{
private:
	std::ostream& os;
public:
	connection_t(std::ostream& os): os(os) {}
	template<typename T>
	void send_message(T& t)
	{
		const std::type_info& info = typeid(T);
		os << info.name() << ":" << t << '\n' << std::flush;
	}

};

int main()
{
	std::ofstream ofs("test.txt");
	connection_t con(ofs);

	int i(35);
	long l(2003);
	float f(5.2f);
	double d(20.5);
	std::string s("hello mom");

	con.send_message(i);
	con.send_message(l);
	con.send_message(f);
	con.send_message(d);
	con.send_message(s);
	con.send_message("what's this");
	con.send_message("And this");

	return 0;
}

Here is the output:


i:35
l:2003
f:5.2
d:20.5
Ss:hello mom
A12_c:what's this
A9_c:And this

[moorecm]

My goodness. I would still just go with the enumerations--they're simpler and more portable.

[jsmith]

+1 moorecm

and use boost::serialization.

[m4ster r0shi]

moorecm wrote:

My goodness. I would still just go with the enumerations--their simpler and more portable.

Was that a reply to me? But I do use integer constants to represent the ids of the data types. I know that in my first suggestion I used strings but that was just something I came up with at that moment because I was in a hurry (I was in the lab pcs and they wanted to close...)

I insist on using polymorphism rather than templates to do this job. You see, when you templetize a function you can't change the behaviour of this function for different types of the template parameter. Well, you can do this with template specialization but then... what's the point of using templates?... It's better to use polymorphism to begin with...

And to help you understand that it is important for the storing/retrieving function to behave differently for some types consider this example:

You have this struct:

struct SomeStruct
{
     vector<int*> pint_list;
     string str;
};

And this struct:

struct AnotherStruct
{
     vector<int> int_list;
     char cstr[50];
};

When you store a SomeStruct item you want to store the value each int pointer of the pint_list list points at (i.e. the object pointed at by the object contained in the vector), while in an AnotherStruct object you want to store the value of the int (i.e. the object contained in the vector). Also while the size of cstr may safely be considered to be 50, you must use str.size() to get the size for str.

jsmith wrote:

use boost::serialization

If you take a look here -> http://www.boost.org/doc/libs/1_43_0/libs/serialization/doc/index.html
you'll see that boost::serialization is also a great way to what you want, as it supports:

4. Deep pointer save and restore. That is, save and restore of pointers saves and restores the data pointed to.

7. Data Portability - Streams of bytes created on one platform should be readable on any other.

But there are more things that you may not need to use... such as:

5. Proper restoration of pointers to shared data.

6. Serialization of STL containers and other commonly used templates.

To sum up, I believe that writing your own interface to do the job, using polymorphism, is the best option you have.

[jsmith]

I have implemented both the inheritance/polymorphic approach (several times) and the serialization approach (once) in my professional career. Looking back, I would never
use the inheritance approach again, for several reasons:

1) It is hard to get right/buggier compared to the serialization approach
2) It requires far more code than the serialization approach
3) It performs far worse than the serialization approach

Just look at your add_to_message and read_from_message functions. There are
push_backs, for loops, computations, etc, etc to store/retrieve relatively simple
data types. Contrast this to boost::serialization where you write ONE function
that performs both the store and retrieve, and said function can be only ONE line of
code.

I also have to reject the following statement as a non-sequitur:

But there are more things that you may not need to use... such as:...

on the simple grounds that one does not use arrays over vectors simply because
the STL contains so much other stuff that you don't need.

[m4ster r0shi]

1) It is hard to get right/buggier compared to the serialization approach
2) It requires far more code than the serialization approach

Of course, since you write your own code! The only thing you see using boost is the interface, the implementation is hidden.

3) It performs far worse than the serialization approach

It does? Well, since you say you've tried both approaches perhaps it does for you. But maybe there are people who can write faster code than boost's... :P

Just look at your add_to_message and read_from_message functions. There are
push_backs, for loops, computations, etc, etc to store/retrieve relatively simple
data types. Contrast this to boost::serialization where you write ONE function
that performs both the store and retrieve, and said function can be only ONE line of
code.

And how does boost::serialization's implementation look like? Again, I'm saying that maybe you're mistaking easy interface with simple implementation... And what's wrong with push_back? It takes almost constant time...

I also have to reject the following statement as a non-sequitur:

Don't! I don't think that the analogy you do here is good... You see, arrays and vectors, and in general STL containers, are merely data-types. But what you see in boost::serialization system goals is abstract ideas... You can't know how exactly they are implemented... Perhaps (similarly to data types) it only takes including/ignoring a header file but perhaps some of them have to be hardcoded somehow thus you have to compile some code that you don't actually use...

[moorecm]

Of course, since you write your own code! The only thing you see using boost is the interface, the implementation is hidden.

The Boost libraries are developed by a community of professionals, code reviewed, tested, and have widely-accepted use.

[m4ster r0shi]

Is the code available? Where can I see it?

[Albatross]

On the Boost website. Duh. :P
http://www.boost.org/users/download/

Look under Subversion Repository.

-Albatross