fwrite cannot wite data into file if data size is very small

[sac]

What's wrong with my following c code? I cant find the problem, Help me!
I am trying to write data into a new file using the fwrite function if data size is small (~ less than 1024 bytes) then file doesn't contain such a data but if data size is very large then file contain the data. Rest of the things goes fine except above problem. I am using Linux 2.6.27.5-117.fc10.i686

Here is code

#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <string.h>
#include <netinet/in.h>
#include <netdb.h>
#include <stdlib.h>
#include <arpa/inet.h>

#include <unistd.h> /* close() */

#define TEMP_CLIENT_DUMP_FILE "client.dat"

#define SERVER_PORT 11111
#define SERVER_IP_ADDRESS   "192.168.1.108"

#define BUFFSIZE 8

void error(char *msg) {
    perror(msg);
    exit(0);
}

int main(int argc, char *argv[]) {

    struct sockaddr_in serv_addr;

    char buffer[BUFFSIZE];
    int socketId, readBytes;
    int n;
    unsigned long totalReadBytes;

    FILE *fp;

    /* Create the socket */
    socketId = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);

    if (socketId < 0) {
        error("ERROR opening socket");
    }

    /* Construct the server sockaddr_in structure */
    memset(&serv_addr, 0x00, sizeof (serv_addr)); /* Clear struct */
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_addr.s_addr = inet_addr(SERVER_IP_ADDRESS); /* IP address */
    serv_addr.sin_port = htons(SERVER_PORT); /* server port */

    if ((fp = fopen(TEMP_CLIENT_DUMP_FILE, "wb")) == NULL) {
        fprintf(stderr, "ERROR, unable to create file\n");
        exit(0);
    }

    /* Establish connection */
    fprintf(stdout, "connectd server %s:%d...\n", inet_ntoa(serv_addr.sin_addr), serv_addr.sin_port/*ntohs(servAddr.sin_port)*/);

    totalReadBytes = 0;
    //while (1) {
    readBytes = bind(socketId, (struct sockaddr *) & serv_addr, sizeof (serv_addr));
    if (readBytes == 0) {
        n = sizeof (serv_addr);
        do {
            readBytes = recvfrom(socketId, buffer, BUFFSIZE, 0, (struct sockaddr *) & serv_addr, &n);

            if (readBytes > 0) {
                
                totalReadBytes = totalReadBytes + fwrite(buffer, 1, readBytes, fp);
                printf("Total read bytes %d \r", totalReadBytes);
            }
        } while (readBytes > 0);
    }else{
        printf("unable to bind\n");
    }
    //}

    close(socketId);
    fclose(fp);

    return 0;
}

[writetonsharma]

does buffer shows it has data?
I mean is it receiving proper data from socket and does the debugger shows that buffer has some data.
how do you know fwrite is not writting data to the file?

[sac]

ya, buffer shows proper data.
I tried for one single "Hello" word i.e. I send this word from server to the client then check the buffer on debug mode. I found proper data. After stop this client I have been checked the size of created file & I found that size was zero as well as I open such file in text editor, there was no data.

But when I sent big data about 1MB then I surprised that a file which is created at client side have a non zero size but size was less than the 1MB & it contain few data which I had send over socket.

[SouravDutta]

I am not sure, but maybe it will write in chunks. So when the data size is smaller than the chunk size it is not writing it to the file. You can try out fflush()-ing the stream onto the file.

[sac]

Now I notes that fwrite write data into the file in block of 4098 bytes. That mean multiple of 4KB. If data shall not be in multiple of 4KB then remaining data is loss and fwrite is not write such data into file.

Is there any characteristics of FILE structure so that we can set block size to write into file??

[writetonsharma]

no fwrite can write an any amount of data to file.. you can write from 1byte to any byte.

[jsmith]

The c++ runtime library is buffering to reduce the number of system calls being made. Add a fflush( fp ); after the fwrite and you'll see the data magically appear.

[writetonsharma]

but he closed his program then also he is not seeing anything in the file!!!

[jsmith]

If by "closing the program" it means pressing CTRL-C or any other means by which the fclose(fp) line does not execute, then it absolutely makes sense.

[jRaskell]

ClientServer programming is inherently complex, and this code is overly simplistic, too simplistic. There's no synchronization, there's no handshaking.

How does the client know that data has been sent from the server? He's setting up a UDP connection, and thus there's no guarantee of packet ordering, or even that a packet that was sent will actually arrive either.

What is likely happening, if you run that code as a client executable, it's going to establish a connection, and check for data received immediately (the check will occur on an order of milliseconds after the executable has run). Even if the server code is set up to initiate the transfer of data immediately upon connection your client code has already executed the first recvfrom and readbytes = 0 on the very first check, and the code falls through without writing any data to file.

When you debug the code, you're probably breaking after connection, thus giving the server time to send data, and then stepping through and grabbing that data out of the buffer. As for the file size... your filepath is relative, not absolute (just "client.dat") the local folder won't necessarily be the same for the executable and the debug environment, depending on your development tools. Set it to an absolute path for testing purposes and check again.

There's also no guarantee that a readbytes = 0 means you've reached the actual end of data transfer. The client code may be pulling the data out of it's local buffer faster than the network can fill it up, and could get an empty recvfrom in the middle of the transfer. This is unlikely on an isolated local network, but far more likely over the internet or a wan.

You should have at least some handshaking between the client and server. Once the client connects, it sends a unique ascii character sequence to the server (this is arbitrary, whatever you want to use) indicating it is ready to receive data. The server returns a separate unique ascii character sequence, indicating data transfer is beginning, followed immediately by the size of the transfer, then the transfer data itself, so the client knows exactly how much data it's receiving.

Even with that bare minimum there's still no guarantee of data integrity with a UDP connection. Using a TCP connection will provide a much more reliable transfer of data, but at the cost of speed. I wouldn't necessarily recommend that route without knowing your end goal though.

[sac]

Thanks jsmith!
Its working now. fflush(fp) doing this job well. Even for one single byte data file contain the data with non zero size.
Thanks again!!

Now writetonsharma, ya, after closing my program I am not able to see data in file if data is less than the 4K else data will be in file but for next data which will be less than 4k block will not present until an unless fflush(fp) not call up.

Now jRaskell, in my case client getting data from server very well and if you see the function which is used to get data from socket on client side is a recvfrom() and I have set flag=0 that means waiting state until an unless data not reach on specific socket. If flag will be MSG_DONTWAIT then you r correct. so kindly look on flag of recvfrom()

Ones again thanks to jsmith.

[writetonsharma]

jsmith:

but he's saying that long data buffer is written to file and short buffer not.. this is strange.
secondly even if we didnt fclose() and exit the program properly, the file will be automatically closed. so that should not be a problem.

jRaskell:

he has a valid file pointer to write to.
he is seeing all the data in the buffer before fwriting it!!!
so if the data is in the buffer why it is not written..???

[jsmith]

@writetonsharma:

Right. fwrite() buffers the short data in an internal buffer held in application space. fwrite() does not invoke the write() system call until fflush(), fclose(), fsync(), or enough data is fwrite()'d to fill the internal buffer. Thus the operating system doesn't know of the data to be written. If the application exits abnormally fclose() never happens. Yes, the operating system does close the file automatically. But since the operating system never saw those few bytes of data (because fwrite() buffered them in application space.) it doesn't think there is anything to write. So yes, indeed, it is a quite a problem.