Main thread is blocking

[Stewbond]

I have an issue where I iterate through devices and make driver API calls. Unfortunately, these calls take too much time and they are interrupting my real-time scheduler. I have 12 cores, of which one is 100% and the others are < 1%.

I'd like to multi-thread this thing.

So far, I've replaced:

for (DeviceIterator d = devices.begin(); d != devices.end(); ++d)
{
  d->Write(words, numwords);
}

with

void* ThreadedWrite(void* p)
{
  WritePackage* wp = (WritePackage*)p;

  wp->device->Write(wp->words, wp->numwords);

  delete[] wp->words;
  delete wp;
  return NULL;
}
...
  for (DeviceIterator d = devices.begin(); d != devices.end(); ++d)
  {
    WritePackage* wp = new WritePackage; // structure
    wp->words        = new int[numwords];
    wp->numwords     = numwords;
    wp->device       = d;
    memcpy(wp->words, words, numwords * sizeof(int));
    CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)ThreadedWrite, (LPVOID)wp, 0, NULL);
  }

My problem is that this didn't improve performance at all. The main thread still takes too long to execute.

Is there something I need to do to prevent the main thread from blocking?

[Disch]

Creating a thread is fairly expensive.

Also... if the words you're writing are the same for all devices, you don't need to duplicate that data for each thread. (Although doing so does save you the trouble of having to guard everything... so pick your poison).

That said... the first solution that comes to my mind is to use a thread pool. IE, you have several threads pre-created that are just sitting around idle waiting for 'jobs'. Then when you want to do something in another thread, instead of creating a new one, you simply hand a job off to the pool, and the pool in turn wakes up one of the idle threads and has it run the job.

There are lots of thread pool libraries around. I know boost has one. But making a simplistic one shouldn't be that difficult if you don't want to add another dependency.

[Stewbond]

Thanks again Disch. The devices actually all use different data. I didn't realize thread creation was an expensive task. Thread-pooling sounds like the right approach to take here (especially since I know during the initialization how many devices I have).