Token Problems

[Trashcan42]

I'm currently trying to learn about using Tokens in basic programs but having a tough time figuring them out. The book I'm learning from is kind of confusing me in a few aspects. What I need help on is:

1. Is "Grammar" a road map used to see where you want the program to go and how it should read things?
2. This code that I'm posting was pretty much taken from the book and said to work, yet when I try and compile it on Dev C++ I get a few errors.

The error I'm getting is it's saying get_token() isn't being declared. Linker error.

class Token{
      public:
             char kind;
             double value;
              Token(char ch)
                        :kind(ch), value(0){}
             Token(char ch, double val)
                        :kind(ch), value(val){}
};

Token get_token();
vector<Token>tok;

int main()
{
    while(cin){
               Token t=get_token();
               tok.push_back(t);
               }
    for(int i=0; i<tok.size(); ++i){
            if(tok[i].kind=='*'){
                                 double d=tok[i-1].value*tok[i+1].value;
                                 }
            }
            system ("pause");
            return 0;
}

Understanding the idea of Tokens and how exactly to implement them is something I need help with. So any type of help would be appreciated.

[Albatross]

1.

Wikipedia editors wrote:

A formal grammar (sometimes simply called a grammar) is a set of formation rules for strings in a formal language. The rules describe how to form strings from the language's alphabet that are valid according to the language's syntax. A grammar does not describe the meaning of the strings or what can be done with them in whatever context—only their form.

2. I don't see get_token()'s implementation anywhere here. Maybe it's being implemented in a different file that's not shown here, but if so then it's definitely not being linked with this one. Is there anything more to this example in your book?

-Albatross

[Trashcan42]

I was under the assumption in my Token get_token(); was implementing it? The example in the book is kind of scattered so it doesn't really show the entire example as I posted it. The way it set up it made

Token get_token();
vector<Token>tok;

these 2 lines global scopes (which I thought was bad).