Splitting off recvfrom's to other threads..

[LuckyIsDog]

Let's say client sends five 2,500 byte sized packets to server in five different threads.

Likewise, the server has an processing thread that reads these packets, detects if they are a file packet then pthread_create's another thread and sends it off for that thread to handle it.

The question is, how to send it to another thread? If all 5 of these threads were running recvfrom() would it not receive whatever the client sent first regardless of split files?

How can we accomplish a task where it would only recvfrom if it can identify the header of the packet to see if that is the packet it is supposed to recvfrom and do work. Then otherwise ignore it and allow the other threads to read that packet?

And by ignore I mean keep the header that it read from the recvfrom instead of claiming that part of the packet already processed -- which in turn would not allow other threads to read.

Kind of stumped here. Any ideas would help!

Here is a link to the open source: https://github.com/luckythedog/udtp/blob/master/src/udtp.cpp#L94
Which you can summarily see my bump in the road.

[kbw]

Let's say client sends five 2,500 byte sized packets to server in five different threads.

I think you're assuming that the server sees 5 reads and each call reads 2500 bytes. That is not the case.

how to send it to another thread?

You don't "send it" to another thread, you have the other thread do the read, or do the read and pass the block to another thread.

would it not receive whatever the client sent first regardless of split files?

It would read whatever turned up first. If you're working on a LAN or localhost, things seem deterministic. But once your stuff is routed across a network with multiple routes like The Internet, those simplistic assumptions do not hold.

How can we accomplish a task where it would only recvfrom if it can identify the header of the packet to see if that is the packet it is supposed to recvfrom and do work. Then otherwise ignore it and allow the other threads to read that packet?

You need to read in one place, decode what you've read and dispatch the blocks to the appropriate handlers ... the Factory pattern.

EDIT: I looked at the code. I'm I don't quite understand it, but it doesn't look ok to me.

I think you should pause that think about the protocol. You need a clear picture of the exchange between the clients and server (which I don't think you have). Then you can begin to implement that protocol in a clear way (also missing).

[LuckyIsDog]

Ahh.. so you're saying just have a one thread read it all in one place much like a supervisor and send it to one of the threads to work on that block.

Can I have the "supervisor" MSG_PEEK the first 5 bytes of the packet in order to find out the identifier then have it open a thread with a recvfrom()

In that "employee" thread can it also have a MSG_PEEK then if it matches the identifier then go ahead and recvfrom() without MSG_PEEK and 2,500 packets?

Is that the correct way of going through the ropes?

[kbw]

Can I have the "supervisor" MSG_PEEK the first 5 bytes of the packet in order to find out the identifier

You have to design that into your protocol. When these comms threads come up, I keep saying protocol, but people just want to get to code.

Have you considered that you may some day want to send a variable length body? The answer is to send a small fixed length header, then read the variable length body.

In your case, the header is also going to have the message type.

You'd also send the data like that, send a small fixed length header followed by the body.

Then you have enough information for the factory to do its job.

[LuckyIsDog]

Have you considered that you may some day want to send a variable length body? The answer is to send a small fixed length header, then read the variable length body.

In your case, the header is also going to have the message type.

Perfect! Just the kind of theorizing I needed. I will try and implement the factory pattern, given that it is the most optimized route of performing multi threaded recvfrom()s. Thanks for your opinion and suggestions towards this project.

[modoran]

Also you should reconsider using a pool of threads (5 in your case) which all run at program start and wait doing nothing then creating and destroying a thread each time as this is going to slow things down quite a bit.

[LuckyIsDog]

Also you should reconsider using a pool of threads (5 in your case) which all run at program start and wait doing nothing then creating and destroying a thread each time as this is going to slow things down quite a bit.

Ahh.. I did not know starting and removing threads slowed things down. But hmm, I was kind of riding on that ability since the API will try and accomodate a certain number of threads depending on bandwidth and power. With more strength, comes more threads.

Maybe I should have 5-10 threads already opened and try creating threads but have a delayed time depending on their last use before they are destroyed?