Tokenizing a string

[pattrick128]

Hi,

Say the input has two lines, both of which are "Hello world". This is the output:

Hello: 6
world
Hello: 11
world: 5

    while (getline(cin, line, ' ')) { //space is the delimiter
        token = line.substr(0, line.find(' ')); //token now has the word separated by space
        count = token.length();
        cout << token << ":" << count << endl;
            
        //erase the part of string that is already tokenized
        line.erase(0, token.length());
        } 
    }

So apparently the last word of the first line isn't getting read, but is being read in the next line. I'm deleting the word that we already tokenized at the end, so that when we substring it, it would always start at 0. Any advice would be nice.

[pattrick128]

Bump

[Bingocat4]

I think this is what you were trying to do

#include <iostream>
#include <vector>
#include <string>
#include <sstream>
#include <Windows.h>

using namespace std;

int main() {
	string input;
	int count = 0;
	vector<string> words;

	getline(cin, input);

	istringstream ss(input);
	string token = "";

	while (getline(ss, token, ' ')) {
		words.push_back(token);
	}
	for (auto word : words)
		cout << word << ": " << word.size() << endl;

	system("pause");
}

[Duthomhas]

Obligatory FAQ
http://www.cplusplus.com/faq/sequences/strings/split/

Use of an std::istringstream is indeed very nice for this kind of thing.