STRTOK and NULL

[karelPrague]

Hello, I am beginner in C++ programming. And i was try use STRTOK code with NULL fields, but in ARRAY fields NULL values is skiped.
For example:

input text in Memo2:

0000||TEXT1|||TEXT2
0002|TEXT3|TEXT4||TEXT5
0003|||TEXT6

AnsiString PAT02[100][20];
for (int IndexVRadku02 = 0; IndexVRadku02 < Memo2->Lines->Count ; IndexVRadku02++)
        {
        AnsiString PNF02 = Memo2->Lines->Strings[IndexVRadku02];
        char * PN02;
        PN02 = strtok (PNF02.c_str(), "|");
        PAT02[IndexVRadku02][0] = PN02;
        for (int AA = 0; AA < 21; AA++)
        {
              PN02 = strtok (NULL, "|");
              PAT02[IndexVRadku02][AA] = PN02;
        }

Result:

Array  00   01     02    03    04    05
  00  0000 TEXT1 TEXT2
  01  0002 TEXT3 TEXT4 TEXT5
  02  0003 TEXT6

But i need this result:

Array  00   01     02    03    04    05
  00  0000       TEXT1             TEXT2
  01  0002 TEXT3 TEXT4       TEXT5
  02  0003             TEXT6

Please help me, thanks!

[Cubbi]

that's a fundamental property of strtok, it treats consecutive delimiters as one delimiter. Consider other tokenizers, for example, boost:

#include <iostream>
#include <iomanip>
#include <sstream>
#include <string>
#include <vector>
#include <boost/algorithm/string/classification.hpp>
#include <boost/algorithm/string/split.hpp>

int main()
{
    std::string in = "0000||TEXT1|||TEXT2\n"
                     "0002|TEXT3|TEXT4||TEXT5\n"
                     "0003|||TEXT6\n";
    std::vector< std::vector<std::string> > out;
    std::istringstream buf(in);
    for(std::string line; std::getline(buf, line); )
    {
        std::vector<std::string> tok;
        split(tok, line, boost::is_any_of("|"));
        out.push_back(tok);
    }

    for(auto& row: out)  {
        for(auto& str: row)
            std::cout << std::setw(10) << s;
        std::cout << '\n';l
    }
}

      0000               TEXT1                         TEXT2
      0002     TEXT3     TEXT4               TEXT5
      0003                         TEXT6

Or use split(vector, string, is_any_of("|"));

Or (if you can't get boost for some reason) call std::getline with '|' as the delimiter in a loop. In short, strtok is not suitable

[karelPrague]

Thanks for your answer.
I was download BOOST headers, but in "mutable_iterator.hpp" compiler (Builder C++ 6) messages me error.

Karel