C++

Forum

 
Tokens

flutok (4)
Ive been going through the book "Programming Principles and Practice using C++ " and got to a problem: When I tried compiling this in Code::Blocks(GNU GCC) it gives an error that get_token() isn't declared. I also don't know what that line is supposed to do. What maybe the source of the problem is that in the book it says I should use <std_lib_facilities.h> and if I don't have it I should download it form the authors website. Thing is the website is apparently down, so if someone could post this file here and also answer the question I would be grateful.


#include<iostream>
#include<string>
#include<vector>
#include<algorithm>
#include<cmath>
using namespace std;
inline void keep_window_open() {char ch; cin>>ch;}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
 
class Token{
public:
    char kind;
    double value;
    Token(char ch)
        :kind(ch), value(0){}
    Token(char ch, double val)
        :kind(ch), value(val){}
}

Token get_token();
vector<Token>tok;

int main()
{
while(cin){
    Token t=get_token();
    tok.push_back(t);
}
}
helios (17574)
flutok (4)
When I try saving that file (on the website) as std_lib_facilities.h, and using #include"std_lib_facilities.h" it just gives me a ton of errors: the first (and last) three are something like 'stray/377' or 376 or # and most of the others are null character, ignored.
And can also someone help and tell me what is wrong with the code itself(that's the main problem)?
Gaminic (1621)
Odd, that seems to be a recent version of the header. Can you copy the exact output?
flutok (4)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
 
C:\xxx\std_lib_facilities.h|1|error: stray '\377' in program|
C:\xxx\std_lib_facilities.h|1|error: stray '\376' in program|
C:\xxx\std_lib_facilities.h|1|error: stray '#' in program|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|1|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|2|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|error: invalid preprocessing directive #d|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|3|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|4|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|5|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|6|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|error: invalid preprocessing directive #i|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|7|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|8|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|error: invalid preprocessing directive #i|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|9|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|10|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|error: invalid preprocessing directive #i|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|11|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|12|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|error: invalid preprocessing directive #i|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
C:\xxx\std_lib_facilities.h|13|warning: null character(s) ignored|
...
C:\xxx\std_lib_facilities.h|1|error: 'i' does not name a type|
C:\xxx\std_lib_facilities.h|43|error: 'u' does not name a type|
C:\xxx\std_lib_facilities.h|47|error: 'n' does not name a type|
C:\xxx\std_lib_facilities.h|67|error: expected unqualified-id before '/' token|
C:\xxx\std_lib_facilities.h|89|error: expected unqualified-id before '/' token|
C:\xxx\std_lib_facilities.h|97|error: 't' does not name a type|
C:\xxx\std_lib_facilities.h|111|error: 's' does not name a type|
C:\xxx\std_lib_facilities.h|123|error: expected unqualified-id before '/' token|
||=== Build finished: 41 errors, 4445 warnings ===|


Its too long to post everything: 4487 lines.
Last edited on
Gaminic (1621)
That... should not happen.
Moschops (7244)
This looks like you have cut n' paste code and in doing so you have included a large number of characters that really shouldn't be there. You probably can't see them - they are non-printing characters, so your editor doesn't show them to you.

If you've cut and paste it from a website, I expect there are all sorts of hidden characters in it.
flutok (4)
Yup, I did that but i tried goggling how to remove those hidden characters and couldn't find anything. I'm using Notepad++ if makes any difference. Actually scratch that, I manged to make it show all characters, and indeed there are some strange characters that i cant seem to mark or select. What to do?
Moschops (7244)
Go back to where you found the original code you copied, and copy it again without all the hidden characters.
modoran (2077)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
 
/*
	simple "Programming: Principles and Practice using C++" course header to
	be used for the first few weeks.
	It provides the most common standard headers (in the global namespace)
	and minimal exception/error support.

	Students: please don't try to understand the details of headers just yet.
	All will be explained. This header is primarily used so that you don't have
	to understand every concept all at once.

	Revised April 25, 2010: simple_error() added
*/

#ifndef H112
#define H112 201004L

#include<iostream>
#include<fstream>
#include<sstream>
#include<cmath>
#include<cstdlib>
#include<string>
#include<list>
#include<vector>
#include<algorithm>
#include<stdexcept>

//------------------------------------------------------------------------------

#ifdef _MSC_VER
#include <hash_map>
using stdext::hash_map;
#else
#include <ext/hash_map>
using __gnu_cxx::hash_map;

namespace __gnu_cxx {

    template<> struct hash<std::string>
    {
        size_t operator()(const std::string& s) const
        {
            return hash<char*>()(s.c_str());
        }
    };

} // of namespace __gnu_cxx
#endif

//------------------------------------------------------------------------------

#define unordered_map hash_map

//------------------------------------------------------------------------------

typedef long Unicode;

//------------------------------------------------------------------------------

using namespace std;

template<class T> string to_string(const T& t)
{
	ostringstream os;
	os << t;
	return os.str();
}

struct Range_error : out_of_range {	// enhanced vector range error reporting
	int index;
	Range_error(int i) :out_of_range("Range error: "+to_string(i)), index(i) { }
};


// trivially range-checked vector (no iterator checking):
template< class T> struct Vector : public std::vector<T> {
	typedef typename std::vector<T>::size_type size_type;

	Vector() { }
	explicit Vector(size_type n) :std::vector<T>(n) {}
	Vector(size_type n, const T& v) :std::vector<T>(n,v) {}
	template <class I>
	Vector(I first, I last) :std::vector<T>(first,last) {}

	T& operator[](unsigned int i) // rather than return at(i);
	{
		if (i<0||this->size()<=i) throw Range_error(i);
		return std::vector<T>::operator[](i);
	}
	const T& operator[](unsigned int i) const
	{
		if (i<0||this->size()<=i) throw Range_error(i);
		return std::vector<T>::operator[](i);
	}
};

// disgusting macro hack to get a range checked vector:
#define vector Vector

// trivially range-checked string (no iterator checking):
struct String : std::string {
	
	String() { }
	String(const char* p) :std::string(p) {}
	String(const string& s) :std::string(s) {}
	template<class S> String(S s) :std::string(s) {}
	String(int sz, char val) :std::string(sz,val) {}
	template<class Iter> String(Iter p1, Iter p2) : std::string(p1,p2) { }

	char& operator[](unsigned int i) // rather than return at(i);
	{
		if (i<0||size()<=i) throw Range_error(i);
		return std::string::operator[](i);
	}

	const char& operator[](unsigned int i) const
	{
		if (i<0||size()<=i) throw Range_error(i);
		return std::string::operator[](i);
	}
};

#ifndef _MSC_VER
namespace __gnu_cxx {

    template<> struct hash<String>
    {
        size_t operator()(const String& s) const
        {
            return hash<std::string>()(s);
        }
    };

} // of namespace __gnu_cxx
#endif


struct Exit : runtime_error {
	Exit(): runtime_error("Exit") {}
};

// error() simply disguises throws:
inline void error(const string& s)
{
	throw runtime_error(s);
}

inline void error(const string& s, const string& s2)
{
	error(s+s2);
}

inline void error(const string& s, int i)
{
	ostringstream os;
	os << s <<": " << i;
	error(os.str());
}

#if _MSC_VER<1500
	// disgusting macro hack to get a range checked string:
	#define string String
	// MS C++ 9.0 have a built-in assert for string range check
	// and uses "std::string" in several places so that macro substitution fails
#endif

template<class T> char* as_bytes(T& i)	// needed for binary I/O
{
	void* addr = &i;	// get the address of the first byte
						// of memory used to store the object
	return static_cast<char*>(addr); // treat that memory as bytes
}


inline void keep_window_open()
{
	cin.clear();
	cout << "Please enter a character to exit\n";
	char ch;
	cin >> ch;
	return;
}

inline void keep_window_open(string s)
{
	if (s=="") return;
	cin.clear();
	cin.ignore(120,'\n');
	for (;;) {
		cout << "Please enter " << s << " to exit\n";
		string ss;
		while (cin >> ss && ss!=s)
			cout << "Please enter " << s << " to exit\n";
		return;
	}
}



// error function to be used (only) until error() is introduced in Chapter 5:
inline void simple_error(string s)	// write ``error: sТТ and exit program
{
	cerr << "error: " << s << '\n';
	keep_window_open();		// for some Windows environments
	exit(1);
}

// make std::min() and std::max() accessible:
#undef min
#undef max

#include<iomanip>
inline ios_base& general(ios_base& b)	// to augment fixed and scientific
{
	b.setf(ios_base::fmtflags(0),ios_base::floatfield);
	return b;
}

// run-time checked narrowing cast (type conversion):
template<class R, class A> R narrow_cast(const A& a)
{
	R r = R(a);
	if (A(r)!=a) error(string("info loss"));
	return r;
}


inline int randint(int max) { return rand()%max; }

inline int randint(int min, int max) { return randint(max-min)+min; }

inline double sqrt(int x) { return sqrt(double(x)); }	// to match C++0x

#endif
Last edited on
Cuddlebuddie928 (30)
That line is supposed to convert input into tokens, but you haven't actually defined it... the header does not include its definition.

See 6.8.1 (Implementing Token_stream) and 6.8.2 (Reading tokens) for the implementation code. If you don't want to implement Token_stream, use the Token_stream::get() function's body as a template for your undefined function... just remove the first 'if' block.
Topic archived. No new replies allowed.