Tokenization of strings
Oct 1, 2013 at 8:42am UTC
I am using the following code which can tokenize the strings based on DELIMITER. But I want this data to be saved in a vector such that I can access the data anywhere in my program. Is there any chance without using vectors I could achieve this? Are vectors really useful?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
#include <iostream>
#include <sstream>
using std::cout;
using std::endl;
#include <fstream>
using std::ifstream;
#include <cstring>
#include <vector>
const int MAX_CHARS_PER_LINE = 512;
const int MAX_TOKENS_PER_LINE = 50;
const char * const DELIMITER = " " ;
int main()
{
ifstream fin;
fin.open("data.txt" );
if (!fin.good())
return 1;
int j=1;
while (!fin.eof())
{
char buf[MAX_CHARS_PER_LINE];
fin.getline(buf, MAX_CHARS_PER_LINE);
int n = 0;
std::vector<std::string> token_1[MAX_TOKENS_PER_LINE] = {};
const char * token[MAX_TOKENS_PER_LINE] = {}; // initialize to 0
token[0] = strtok(buf, DELIMITER);
if (token[0])
{
for (n = 1; n < MAX_TOKENS_PER_LINE; n++)
{
token[n] = strtok(0, DELIMITER);
if (!token[n]) break ;
}
}
for (int i = 0; i<n; i++)
cout << "Token[" << j++ << "] = " << token[i] << endl; // token[i], this I need to access anywhere in my program.
}
cout<<--j<<endl;
}
I want token[i] to be accessed for suppose if I give token[10], I could be able to get the value of that particular token.
Oct 1, 2013 at 8:45am UTC
But I want this data to be saved in a vector such that I can access the data anywhere in my program. Is there any chance without using vectors I could achieve this?
No. There is no way you can save stuff in a vector without using a vector.
Oct 1, 2013 at 8:48am UTC
Well, thanks a lot @cire, how can I use vector to save token[i]? I cannot copy/ assign , it gets lots of errors!
Oct 1, 2013 at 10:16am UTC
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
#include <iostream>
#include <vector>
#include <string>
void anywhere_in_this_program( std::vector<std::string>& tokens)
{
for (auto &t : tokens)
std::cout << t << '\n' ;
}
int main()
{
std::vector<std::string> tokens;
std::string token;
while (std::cin >> token)
tokens.push_back(token);
anywhere_in_this_program(tokens) ;
}
http://ideone.com/QOvEl9
Topic archived. No new replies allowed.