Send binary data over TCP/IP

[simptri69]

Hello everyone. I am new to this forum and this is my first question so my apologies if it doesn't meet the standards.

I am writing a client-server program in C++ using C libraries. The server listens for a connection and when one is found, it is supposed to send a binary file(an image in my case) to the client. The server is sending the binary file perfectly after opening it and reading its contents into a buffer that is dynamically allocated in the SERVER file.

The problem begins when the client file is supposed to recv() the file, I do not know how I can allocate a buffer big enough for the file to be received. I know how to use malloc() and new, I prefer malloc() for executable size customization. Assuming the file being sent is 11000 bytes(note that this is an assumption and the client can not know the file size because the file size is dynamically generated by server).

How can the client generate a dynamically allocated buffer big enough to hold 11000 bytes?

OR

How can the client store the data in a buffer using recv() without knowing the file size.

Below is part of the code that does the send()'ing and recv()'ing.

SERVER

FILE *img = fopen("img.jpg", "rb");
fseek(img, 0, SEEK_END);
unsigned long filesize = ftell(img);
char *buffer = (char*)malloc(sizeof(char)*filesize);
rewind(img);
// store read data into buffer
fread(buffer, sizeof(char), filesize, img);
// send buffer to client
send(clientsocket, buffer, filesize, 0); // error checking is done in actual code and it sends perfectly 

CLIENT

// client recv()'s file, how do I generate **filesize** here?
// recv(clientsocket, buffer, strlen(buffer), 0);
// then printf("%s", buffer) to console in client
// if I malloc() small amount to buffer, it outputs all the binary code
char *buffer = malloc(sizeof(char)*5);
recv(clientsocket, buffer, sizeof(buffer), 0);
printf("%s", buffer);
// this outputs all the binary code perfectly but it is a bug because it can't be written to a file
// is there a way to dynamically increase the buffer until all filesize is received?
// certainly, buffer needs to be dynamically allocated or program crashes 

Also one more thing, when I telnet the server from command line, telnet displays all the output perfectly as well

Maybe telnet is storing data into a buffer behind the scenes, if so, how?

[a k n]

A simple solution could be that the server sends the file size first to the client so that client can allocate the sufficient memory required and then the server could send the original file.
Alternatively you could set a limit say ~1K of data to be sent/receieved at one time , so you know that the data could be at-most your limit , for files larger than your limit you split that file and send the parts.
However , I am no expert in networking &c , there could be some other better ways of doing it or a way to know the packet size.

[simptri69]

I am currently working on the first solution, it did occur to me mind but I thought that I should ask if there's a better way, turns out that I will have to try the first solution.

The second solution also came to me mind but I have no idea how to send the file in bits, could you post some code or refer me to a link where you can not necessarily send() a file but rather split a file in C(not C++, don't wanna use the libraries) and allocate memory for the split sizes then combine them together. I hope you get what I mean.

[a k n]

combine them together

In the example code , you just print the data , in such a trivial case you don't have to combine the data.

Sending a file by splitting could be done like this:

//for some MAX_DATA_LIMIT,
if(filesize < MAX_DATA_LIMIT)
        send(clientsocket, buffer, filesize, 0);
        //assuming bytes sent == filesize
    else
    {
        int bytes_sent = 0;
        while(bytes_sent < filesize )
        {
            bytes_sent += send(clientsocket, buffer+bytes_sent, MAX_DATA_LIMIT-1 , 0);
        }
    }

Similarly , you can set the socket to non-blocking and recv() the data in a loop unless it returns -1.
Combining data could be inefficient and slow , so you should instead just use the first method if you want all the data together.

[simptri69]

and here I assume that the buffer is allocated MAX_DATA_LIMIT bytes?

and the client will also do the same for receiving the bytes, i presume?

one more thing, the 2nd parameter to send() must be a const char*, yours happens to be an int if i'm not mistaken.

[a k n]

Yes*.
Yes*.
I'm adding an offset to a pointer to char,I get a pointer to the data at an offset bytes_sent*sizeof(char) bytes from the start of the buffer.
But , I suggest you use the first solution , allocating more space and concatenating new data would be SLOW and ugly.
EDIT:
* : It doesn't even matter if MAX_DATA_LIMIT is different for both server or client , in fact you don't need to split the file in the server since with TCP data is just a continuous stream of bytes, you only have to modify your client to only take in MAX_DATA_LIMIT , realloc() and in the next recv() cycle , add data after an offset.

[simptri69]

I'm gonna post all of my C/C++ questions here, at least I get the help that I'm looking for and the question doesn't get voted down and bull**** like that.

Thanks a k n

[fcantoro]

In general, you should send a header before the data.
In your case, the header contains at least the data length.
The server send() the header and then send() the data;
The client recv() the header, sets the data buffer and then recv() the data.

Server example, without error checking, based on your code:

struct header
{
	long	data_length;
};

FILE *img = fopen("img.jpg", "rb");
fseek(img, 0, SEEK_END);
unsigned long filesize = ftell(img);
char *buffer = (char*)malloc(sizeof(char)*filesize);
rewind(img);
// store read data into buffer
fread(buffer, sizeof(char), filesize, img);
// send header to client
header hdr;
hdr.data_length = filesize;
send(clientsocket, (const char*)(&hdr), sizeof(hdr), 0);
// send buffer to client
send(clientsocket, buffer, filesize, 0); // error checking is done in actual code and it sends perfectly 

Client example, without error checking, based on your code:

struct header
{
	long	data_length;
};

header hdr;
// receive header
recv(clientsocket, (const char*)(&hdr), sizeof(hdr), 0);
// resize buffer
char *buffer = malloc(sizeof(char)*hdr.data_length);
// receive data
recv(clientsocket, buffer, sizeof(buffer), 0);

[simptri69]

Nice answer fcantoro.

It's almost the same as sending the filesize first. The first option that a k n suggested. That's some good piece of code, it helps a lot.

Hope it works because you are casting struct header to const char* in the send(), recv().

[kbw]

You need that loop with the send too.

FILE *img = fopen("img.jpg", "rb");
fseek(img, 0, SEEK_END);
unsigned long filesize = ftell(img);
char *buffer = (char*)malloc(sizeof(char)*filesize);
rewind(img);
// store read data into buffer
fread(buffer, sizeof(char), filesize, img);

// send buffer to client
int sent = 0;
while (sent < filesize)
{
    int n = send(clientsocket, buffer + sent, filesize - sent, 0);
    if (n == -1)
        break;  // ERROR

    sent += n;
}