Tokenizer for calculator

Hello. I am coding tokenizer which split expression( for example 123+45 123, + , 45)for calculator. It works correctly excpet float/dobule numbers beacuse I dont know how to put dot between digits.
What do you think about this code?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
 vector<Token> MathFunctions::Tokenization(string exp)
{
    int j;
    string tmp;
    vector <Token> tokens;
    for(int i=0; i<exp.size(); i++)
    {
        if(isdigit(exp[i]))       
        {
            tmp = exp[i];
            j=i+1;
            while(isdigit(exp[j]))
            {
                tmp += exp[j];
                j++;
            }

            cout<<tmp<<endl;
            Token tok(tmp,false,true,false);
            tokens.push_back(tok);
            i = j-1;
        }
        else if(exp[i]=='+'||exp[i]=='-'||exp[i]=='*'||exp[i]=='/')
        {
           if(exp[i]=='-'&&exp[i-1]=='\0'||exp[i-1]=='(')
           {

                tmp = exp[i];
                 j=i+1;
              while(isdigit(exp[j]))
            {
                tmp += exp[j];
                j++;
            }
            cout<<tmp<<endl;
            Token tok(tmp,false,true,false);
            tokens.push_back(tok);
            i = j-1;

           }
          else
           {
            cout<<exp[i]<<endl;
            tmp = exp[i];
            Token tok(tmp,true,false,false);
            tokens.push_back(tok);
           }

        }
        else if(exp[i]=='('||exp[i]==')')
        {
            tmp = exp[i];
            Token tok(tmp,false,false,true);
            tokens.push_back(tok);
        }

   }
    for (vector<Token>::iterator it = tokens.begin(); it != tokens.end(); ++it)   
    cout <<" "<< (*it).TokenName;

    return tokens;
}
Lines 11 & 12: what if you hit the end of the exp?
Line 25: what if you get here when i==0?
Lines 29 & 30: same comment as lines 11 & 12

As for doubles, after parsing a sequence of digits, check if the next character is a '.'. If so, add it, and any digits that follow it, to the token.
I found it easier to handle everything else, then what is left is either a number or an error.
I did that with a switch default but you can do with if/else ... the final else would be what is left, and check if its a valid number in there. Don't forget to figure out a scheme for when - is subtraction and when - is part of a negative number. I did that by saying if the token is length 1 and "-" then its subtraction else its part of number (or error).
dunno if this helps you or not.
Topic archived. No new replies allowed.