Difference between token and literal

Apr 22, 2009 at 10:22am
Hi all,
Can any one tell me the difference between token and literals and how they are related to eachother.

Thanks
Suvojit
Last edited on Apr 22, 2009 at 10:22am
Apr 22, 2009 at 10:30am
A token is a categorized block of text. The block of text corresponding to the token is known as a lexeme. A lexical analyzer processes lexemes to categorize them according to function, giving them meaning. This assignment of meaning is known as tokenization. A token can look like anything; it just needs to be a useful part of the structured text.
Example: "res = foo + ++bar * 4;" has the tokens "res", "=", "foo", "+", "++", "bar", "*", "4", and ";".

In computer science, a literal is a notation for representing a fixed value in source code. Almost all programming languages have notations for atomic values such as integers, floating-point numbers, strings, and booleans; some also have notations for elements of enumerated types and compound values such as arrays, records, and objects.
Examples: 67, "foo", '\\', 0x0A.

So, a literal is a type of token.

http://en.wikipedia.org/wiki/Token_(parser)
http://en.wikipedia.org/wiki/Literal_(computer_science)
Last edited on Apr 22, 2009 at 10:31am
Apr 27, 2009 at 9:30am
hi
thanks....
but can u please explain me the literal in a bit simple term( I found it a bit technical).
I am sharing my understanding :

token: Any recognized block of text in the source code. It consists of atomic parts known as lexeme which are processed by the lexical analyzer like lex on basis of their type and in the end the whole block gets a meaning.

Literals: It is used to express the values in code.

Can you tell me if i am correct or not?

Also can you give me some example of literals of enumerated type

Thanks
Topic archived. No new replies allowed.