Tokens

[yayo]

Hallo,

I try to read the content of a file and get the tokens from the file. I already implemented so that I got the tokens from the file but the first token was #include and I want to split this token again because I want # as a token and include as other token. So I implement another method to analyze the tokens

my code to get the tokens from file.

while (input >> nextToken)
		{
			cout << "Token: " << nextToken << endl;
			analyseToken(nextToken);
		}

to analyse tokens:

void analyseToken(string token)
{
	const char* p = token.c_str();
	
	while (*p != '\0')
	{
	  cout << "char: " << *p++ << endl;	
        }
}

how can I get the second char from token? example the token #include. I want to get the first char which is # and save it in a buffer and get the rest token which is include and save it in other buffer.

can somebody give me a tips? I'm new in C++ :(

thanks
Yayo

[strongdrink]

substr, my friend.
http://www.cplusplus.com/reference/string/string/substr/

[bluecoder]

ok . i think you need this .. as strongdrink suggested ..

#include <iostream>
#include <string>
using namespace std; 

int main()
{
	string str = "#include"; 
	string firststr , secstr ; 
	size_t pos = str.find("#");
	firststr = str.substr( 0 , pos + 1 ) ; 
	secstr = str.substr(pos +1);
	cout <<"\n The first string  = "<<firststr;
	cout<<"\n The second string = "<<secstr;
}

[yayo]

thanks for your help. but unfortunately it doesn't help me more. I want to find three kinds of tokens in a file

1. word and word that beginning with _ e.x. (test, _test, this_is_a_test, test001)
2. numbers (9, 9.87)
3. other symbols (!"§$%&/()=?)

any ideas???

thanks
Yayo

[moorecm]

There are many ways to solve this type of problem. One of the easier ways is using regular expressions (Boost.Regex is one such library).

However, if you really want to do this right, consider using Boost.Spirit or some other library that let's you define a syntax and then perform the lexical analysis.

Here are some example applications. If you're parsing C++, definately check out cpp_to_html:
http://boost-spirit.com/repository/applications/show_contents.php

[yayo]

Hallo,

I already have a method to read each carachter in a file. using for loop with a variable int c. My problem now It doesn't read the charachter underline " _ ".

I just try this but unfortunately it doesn't work.

if (c == 95) // 95 ANSII its _
cout << "it is underline" << endl;

And it works with the other ANSII code.

can somebody explain to me why? and how can I get the character 95 (underline) in ANSII table?

thanks for help
Yayo

[Vins3Xtreme]

Use a variabile 'char c' and compare its value with '_' .

[yayo]

thanks for your answer. I already do that with char c but still doesn't work :(.

if (c == '_') // 95 ANSII its _
cout << "it is underline" << endl;

strange