Sockets and threads: problem with recv

[flyontheweb]

Hi forum!

I'm quite new to this sockets and threads stuff, so excuse me in advance if this is dumb question. But I've been testing and googling for a couple days and still don't figure out what I'm I doing wrong.

The situation:

I want to wrap a call to recv inside a thread so if for some reason the server doesn't respond in n seconds I can kill the thread and inform the user.

I've put together all data I think I need about the socket in a struct, so I can pass a reference to it to my controlling function:

typedef struct THREADSOCKETIO {
	SOCKET socket;		// socket handler
	char * data;		// send/recv buffer
	int length;		// buffer size
	int ret_length;		// value returned by send/recv
	BOOL done;		// flag to indicate send/recv has returned
} THREADSOCKETIO;

Now my controlling function (I'll show recv only for the shake of brevity)

UINT CReaderLogic::SocketRecvThread( LPVOID lParam )	// yep, this is static
{
	THREADSOCKETIO *params = (THREADSOCKETIO *)lParam;
	params->ret_length = recv( params->socket, params->data, params->length, 0 );
	params->done = TRUE;
	return 0;
}

And finally a code snippet of the calling routine:

//...
CWinThread *sockThread;
DWORD threadStatus;
BYTE ucpResponse[520];
int length = 520;
int retlength;
THREADSOCKETIO *sockParams = new THREADSOCKETIO;

sockParams->socket = sock;		// valid initialized socket handler
sockParams->data = (char *)data;	// buffer where I want the received data
sockParams->length = length;		// buffer size
sockParams->done =  FALSE;		// the "I'm done" flag
// Launch thread: 
sockThread = AfxBeginThread( SocketRecvThread, sockParams );

// point (a), for later reference

// This is my oh-so-simple attempt to controll that recv
// doesn't take more than <timeout> millisecs
int attempt = 0;
while ( !sockParams->done && ( attempt++ < timeout ) ) {
	Sleep( 1 );
}

// point (b), for later reference

if ( !sockParams->done ) {
	// killing thread code I don't want to bore you with
	// irrelevant to the problem as the thread does exit normally
}

retlength = sockParams->ret_length;
//... 

Where's the problem? retlength has less bytes than expected, moreover if I show more than retlength bytes from data buffer I see that actually there are only retlength valid bytes, and the rest is garbage (0's to be exact).
But if I put a Sleep(20) anywhere between (a) and (b), voilá! all expected data is there.

Why???

Is it possible that my "flag" var raises before recv finishes?
Is it posiible that not all expected data is available after recv finishes?

Any comment will be appreciated. Thanks in advance for your time

[writetonsharma]

Where's the problem? retlength has less bytes than expected, moreover if I show more than retlength bytes from data buffer I see that actually there are only retlength valid bytes, and the rest is garbage (0's to be exact).
But if I put a Sleep(20) anywhere between (a) and (b), voilá! all expected data is there.

ret_length is showing less bytes than what you should get.. rest i am not able to understand whats the problem.

Suggestion apart from your problem:
why dont you use, WaitForSingleObject instead of looping and sleeping..

[flyontheweb]

The problem is no less than the server is actually sending MORE bytes than I'm receiving. I mean, while testing I know in advance what am I supposed to receive. And it only works when I place a little Sleep(n) where I mentioned. Of course this is not a solution, this is not even a patch.

I'll try the WaitForSingleObject. Thanks for answering! ;-)

[jsmith]

You are using a TCP socket or a streaming UDP socket, correct? If so, TCP sockets are stream-oriented, in which case this is exactly expected behavior and you have to loop until you receive the number of bytes you want.

[flyontheweb]

thanks JSmith. you're right, I forgot to specify that I'm using a TCP socket.

I know recv could return only a part of the whole incoming data, but this is not the problem. by now I'm sure that I'm receiving much less than the 512 I send to recv as buffer length. the question is that if I'm receiving, let's say 32 bytes and I don't place a little sleep, like Sleep(20) or so, I'm receiving only part of those 32 bytes.
What is more strange, in a subsequent send/request I'm receiving the missing bytes first instead of just the bytes corresponding to the new server response. It sounds like at a lower level there was a system buffer that keeps all incoming bytes but makes wrong on saying where a particular datagram begins or ends... does it make sense to you?

Thanks for posting

[kbw]

When reading sockets, you should read in a loop until you know you have all the data, or are intentionally ignoring what's on the stream. TCP protocols tend to have a sentinel, a known number of bytes or an end-of-line or something, otherwise you won't know when to stop reading.

[writetonsharma]

What is more strange, in a subsequent send/request I'm receiving the missing bytes first instead of just the bytes corresponding to the new server response

This means that the sender is not sending the whole data in single iteration... thats why when you iterate again for some other data you receive the old data.. as i havent seen the sender code i can't say correctly whats wrong. you can post the sender code also.

but this is strange that after putting sleep you gets all the data..

[flyontheweb]

That's the point writetonsharma! Why the Sleep?

Now I'm re-reading some tutorials, wondering where did I get the idea, a wrong idea I guess, that if the sender was not sending the whole data,
int recv(int sockfd, void *buf, int len, unsigned int flags);
would return the same value than param len. I know I read it, but I can't remember where...

Anyway I changed my dumbass waiting loop for a WaitForSingleObject (sorry I'm quite new to this), and still the same problem and the Sleep stuff... Why would the thread terminate before recv has received the whole data? Why it gets "fixed" with a sleep? Maybe something to do with non-blocking recv call? Is that the default for Winsock?

I'll keep researching and post back if I find something.

Thanks a lot guys!

[kbw]

Just because the sender sends a block of data in a single call, doesn't mean that the receive can read the data in a single call. The network can break up the block if it needs to.

Is your thread reading in a loop until it knows it's received all the data?

[flyontheweb]

Now I see my mistake was there kbw.
Is recv suposed to just return 0 if the server has nothing to send? Otherwise I don't have any expected data length beforehand

[kbw]

If it's asyncronous, it will. If it's synchronous, it usually blocks until it receives something.

[flyontheweb]

I've been reading some more about Winsock...

http://msdn.microsoft.com/en-us/library/ms740506(VS.85).aspx
http://msdn.microsoft.com/en-us/library/ms740121(VS.85).aspx

...but still can't figure out how to know if my socket (or it depends on the recv call?) is synchronous or asynchronous. I'm creating the socket like this:
socket( AF_INET, SOCK_STREAM, 0 )

Any idea?

[kbw]

It's synchronous by default. You have to do stuff to make it asynchronous.

[DaveMortimer]

Remember that the power of threading lies in the fact that each separate thread has its own processor. However, a thread's processor will probably not be a separate *physical* processor because there will probably not be enough physical processors to go around for the number of threads you need to run simultaneously, and so therefore the OS emulates multiple processors using time allocations on the main processors.

In the code you have shown us, you have two threads - the main thread T1 and a delegated thread T2 which you want to do the donkey work. When you use a short sleep, T1 is not giving T2 enough time to receive the incoming data before T1 terminates T2 and reads the number of bytes that T2 has received.

The problem lies in the design of your program. At line 27, if done == false, then it is most likely that all the data will not have been received, and you terminate T2. If done is true, then the data has been received (hopefully) and you do not terminate T2.

I understand that the overall design of your program is meant only for learning purposes, but the design is poor. I think your design would be much neater if it used a semaphore class to start and stop the threads when required. Or, you may even be able to have T2 stop T1 while it receives the data, then start it again after.

Also, rather than interacting directly with any OS socket API, you will be much better off in the long run if you invest some time into learning how to use a good network programming library such as...

http://www.cs.wustl.edu/~schmidt/ACE.html

Learning how to use this library will allow you to concentrate on networking, and you can almost forget about all the frustrating idiosyncrasies of all the different OS socket API's. There are also many other classes including threads and semaphores.

Back to your problem. Bearing in mind that threads execute concurrently, consider the following scenario...

(1) T1 starts T2 and delegates T2 to receive 520 bytes.
(2) T1 continues on to the while loop, and T2 begins receiving data, done == false.
(3) T1 reaches the if-statement and reads the value of done, finds it false because T2 is still receiving, and terminates T2 before it is "done".
(4) So now T2 has received 520 minus n bytes, and T1 has reached the point where it reads the value of ret_length.

So if T1 waits long enough (with a longer sleep), then it will find out about all the bytes sent. But if T1 does not sleep for long enough, then it will only know about the number of bytes that T2 has actually received.

Dave

[flyontheweb]

Thanks for your advises Dave. Since the first post I've learnt a little about threads, using CEvent for syncronization and so on... but back to the old dirty code, I still dont get what was going on. Because in your scenario (3) never happens. I mean, done is true and T1 never terminates T2 (we're talking about 30-40 bytes here in a local machine cl/srv scenario).

It's like in T2 recv says "ok, I've received 4 bytes, you have them in the buffer and I'm done", then I look in the buffer, I see there's only those 4 bytes BUT I know for sure that server is sending more. Remember with a Sleep(20) I get all of them, i.e. recv seems to behave different. So... what's going on here?

I was wondering that maybe recv does something like:
(1) reception has finished and data is in an internal buffer.
(2) it spawns a thread (or kind of) that copies data from system buffer to my data var and updates length var according to it.
(3) meanwhile T2 goes ahead, reaches done = TRUE and terminates.
(4) T1 exits while loop, skips if conditional as done == TRUE and ends up while this supposed "copy thread" is still working.

I'm just fantasizing here... Does it make any sense to you?

Thanks for your time

[DaveMortimer]

I am not interested in your fantasies unless they involve Judge Judy and spanking.

done is false as T2 is receiving. While T2 is receiving, T1 is busy moving from the point where it started T2. T1 and T2 are executing concurrently. Think of them as two different processors (virtually at least), and unless you tell them to communicate, they will carry on doing their own things independently. So, while T2 is still receiving and done is still false, T1 has moved on to evaluate the condition...

(!done) = (!false) = true

...and therefore terminates T2 while T2 is still receiving.

But when you suspend T1 by putting it to sleep for long enough, T2 keeps going and has time to receive all the data before T1 can test done. T2 sets done to true, and in this case the if-condition !done = !true = false, and so T1 does not terminate T2, but this makes no difference anyway.

What you have here is a kind of 'race condition'. T1 and T2 are 'racing' each other. When you suspend T1, you allow T2 to race ahead of T1 and receive all the data.

Dave

[flyontheweb]

Paraphrasing Asimov, your logic is undeniable: (!done) = (!false) = true
But your rush is bigger than your logic:

while ( !sockParams->done && ( attempt++ < timeout ) ) {
	Sleep( 1 );
}

Means:

while ( true ) {
	Sleep( 1 );
}
// let's assume I've tried huge timeouts so (attempt++ < timeout) is true till next Easter 

In other words, wait, wait and wait...
So by now I'll go on fantasizing. You can keep Judge Judy... but please don't tell us what u do on her :-S

Thanks again for your time

[DaveMortimer]

Hi flyontheweb.

I was thinking about your problem within the context of the information you initially gave us. You did not give us the value of timeout, so therefore I ignored the while loop since its condition made no sense without this value. (I thought you had omitted this information deliberately.)

In your last post you gave us more information, you said...

// let's assume I've tried huge timeouts so (attempt++ < timeout) is true till next Easter

I had reasonably assumed that timeout was low enough to force the while loop condition to evaluate to false before T2 had received all the data.

By the way, did you find out why you were having problems?

Dave