Basic performance testing

[MrHutch]

Hi all,

Just wondering what people generally use for small speed tests. I've just been testing the odd thing (whether iterators are faster than overloaded operators for the STL vector, that sort of thing). I've been using the clock method of time.h.

Anyone know of any other approaches to this?

For the records, here's a short example of the sort of thing I've been doing.

#include <iostream>
#include <vector>
#include <time.h>

int main(int argc, char* argv[])
{
   const int NUM_ELEMENTS = 5000000;
   int clicks, lc, curr;
   float secs;
   vector<int> vec;

   // Push values to vector
   for(lc = 0; lc < NUM_ELEMENTS; lc++)
   {
      vec.push_back(lc);
   } // for

   // Access and assignment (using [] operator)
   for(lc = 0; lc < NUM_ELEMENTS; lc++)
   {
      curr = vec[lc];
   } // for

   clicks = clock();
   secs   = ((float)clicks)/CLOCKS_PER_SEC;
   cout << "Operation took " << clicks << " clicks (" << secs << " seconds)\n";

   return 0;
} // main 

I'm aware that this isn't a true representation of access time, as it measures the whole program, including allocation and assignment of vector elements. I was thinking I could use time rather than clocks to measure it.

Just wondered out of curiosity what others use.

[Cubbi]

You should be familiar with profilers just as you're familiar with debuggers and compilers. Depending on the development platform, you may either need a special compiler switch, or a special library, or nothing at all (AIX is cool like that), to create an executable that you can then run with the profiler and gather detailed statistics on execution time of every function.

But, to answer the question directly, at home, I use std::high_resolution_clock, as in

    auto t1 = std::chrono::high_resolution_clock::now();
    test_function();
    auto t2 = std::chrono::high_resolution_clock::now();
    std::cout << "test function took "
              << std::chrono::duration_cast<std::chrono::milliseconds>(t2-t1).count()
              << " milliseconds\n";

At work, we have a stopwatch class in our corporate library that tracks user, cpu, and system time simultaneously.

In trivial cases like this, I find that it's simpler to compile the program to assembly language (option -S on gcc and many other compilers), and compare the produced machine code. For the array vs vector access loop, you'll find out that it is exactly the same - once you build a valid test case, of course. Your test loop does nothing and is removed completely by modern compilers)