- Forum
- General C++ Programming
- consumer & producer threading model
consumer & producer threading model
Hi,
Here is a common scenario I often face and would like to know your suggestion for the best approach.
It is a very simple consumer and producer patten. I have a producer thread getting data from tcp socket, package it to a stl queue. Consumer thread will keep checking the queue, dequeue and process the data.
As I cannot lock up the tcp sender, so producer should process as fast as possible even though that cause queue growing up.
I have two approaches
1. the consumer thread keeps dequeue as long as there is element in the queue. Sleep 1 ms only if it is empty. (dequeue and enqueue is protected by mutex)
2. use a condition variable (as in boost), consumer thread will wait if empty. Consumer and producer will communicate by signal.(using boost condition variables)
I used 1 a lot due to the concern that the approach 2 may unnecessarily block the producer. And it seems expensive to keep calling notify for each enqueue. However, thread sleep looks not a good solution at all, though it works so far.
Any suggestion a better third approach?
Thanks a lot
chrisben
Here is a common scenario I often face and would like to know your suggestion for the best approach.
It is a very simple consumer and producer patten. I have a producer thread getting data from tcp socket, package it to a stl queue. Consumer thread will keep checking the queue, dequeue and process the data.
As I cannot lock up the tcp sender, so producer should process as fast as possible even though that cause queue growing up.
I have two approaches
1. the consumer thread keeps dequeue as long as there is element in the queue. Sleep 1 ms only if it is empty. (dequeue and enqueue is protected by mutex)
2. use a condition variable (as in boost), consumer thread will wait if empty. Consumer and producer will communicate by signal.(using boost condition variables)
I used 1 a lot due to the concern that the approach 2 may unnecessarily block the producer. And it seems expensive to keep calling notify for each enqueue. However, thread sleep looks not a good solution at all, though it works so far.
Any suggestion a better third approach?
Thanks a lot
chrisben
You don't give enough information about your circumstances to really help. If you are running on a typical desktop or server platform, I'm a bit confused by some of your concerns. If this is in regards to a specialize platform, embedded or high-performance cluster, knowing that would help.
Unless your CPU is rather underpowered compared to the network interface, you should easily be able to handle the receiving and enqueueing the data without worrying about the time spent blocking on a condition variable. On a typical server OS, the kernel is buffering TCP data.
You really do need to limit the size of the queue to prevent running out of memory. That limit may be very large, but there should at least be one.
Another option is to create a real thread-safe queue that supports multiple producers and consumers and which handles all of the synchronization/wait/wake semantics internally. It's a damned shame that Boost doesn't have one. But they are pretty easy to write using Boost threading primitives.
If you really only have one producer and one consumer, this may be useful: http://msmvps.com/blogs/vandooren/archive/2008/12/02/470800.aspx
Unless your CPU is rather underpowered compared to the network interface, you should easily be able to handle the receiving and enqueueing the data without worrying about the time spent blocking on a condition variable. On a typical server OS, the kernel is buffering TCP data.
You really do need to limit the size of the queue to prevent running out of memory. That limit may be very large, but there should at least be one.
Another option is to create a real thread-safe queue that supports multiple producers and consumers and which handles all of the synchronization/wait/wake semantics internally. It's a damned shame that Boost doesn't have one. But they are pretty easy to write using Boost threading primitives.
If you really only have one producer and one consumer, this may be useful: http://msmvps.com/blogs/vandooren/archive/2008/12/02/470800.aspx
Thanks PanGalactic. Your link is very interesting. I was researching nolock multithreading approach and this is a good article I can study.
I am using common servers (LINUX or windows). However, in my situation, my CPU can be overwhelmed by the network interface, since part of CPU need to process data. Incoming data can have spikes, at that point there is no way the processing thread can work faster than the incoming data. Therefore, I did not consider a condition variable when I implemented it. Even though server will buffer TCP data, it may overflow and slow down the sender momentarily, which I have to avoid since the sender need to service multiple clients. Yes, I am now thinking maybe locking conditional variable is so bad, but wondering whether others had dealt with the similar situations and use it successfully.
I am using common servers (LINUX or windows). However, in my situation, my CPU can be overwhelmed by the network interface, since part of CPU need to process data. Incoming data can have spikes, at that point there is no way the processing thread can work faster than the incoming data. Therefore, I did not consider a condition variable when I implemented it. Even though server will buffer TCP data, it may overflow and slow down the sender momentarily, which I have to avoid since the sender need to service multiple clients. Yes, I am now thinking maybe locking conditional variable is so bad, but wondering whether others had dealt with the similar situations and use it successfully.
hmmm...volatile is a compiler dependent...so be careful
If you use lock free queues, what will the consumer thread do when the queue is empty? Will it chew 100% waiting for a message to arrive?
If you use lock free queues, what will the consumer thread do when the queue is empty? Will it chew 100% waiting for a message to arrive?
Last edited on
Hi,
another approach: Take a classical semaphore, which has a counter. Every release/unlock increases the counter by one, every consumption/lock will decrease the counter by one.
Whenever the producer thread has put something into the queue, it will release the semaphore. This does not block the thread (or only for a very short time in order to synchronize the semaphore itself).
Whenever the consumer thread is ready to consume another thing from the queue, it simply locks the semaphore one more time. If there is something in the queue (i.e. the semaphore counter is greater zero), this lock will be successful and something can be removed from the queue. If the queue is empty, the consumer will block and wait for another package.
I don't know if there is an implementation of such a semaphore out there, but probably there is, since these semantics are standard.
Greetings, TheBear
another approach: Take a classical semaphore, which has a counter. Every release/unlock increases the counter by one, every consumption/lock will decrease the counter by one.
Whenever the producer thread has put something into the queue, it will release the semaphore. This does not block the thread (or only for a very short time in order to synchronize the semaphore itself).
Whenever the consumer thread is ready to consume another thing from the queue, it simply locks the semaphore one more time. If there is something in the queue (i.e. the semaphore counter is greater zero), this lock will be successful and something can be removed from the queue. If the queue is empty, the consumer will block and wait for another package.
I don't know if there is an implementation of such a semaphore out there, but probably there is, since these semantics are standard.
Greetings, TheBear
Topic archived. No new replies allowed.