C++

Forum

 
 
crash in getline loop

poiioppoiiop (9)
hey guys, im helping a friend to get a database for his thesis. it is about chess.
We found a chess database, and with the program that is delivered with it, there is a function to output the 5,7 mln games database as a PGN file. but because there is a 2gb file limit, you can only make chunks of the database of 1.2 GB (about 25% of 5,7 mln games)
it looks like this(and than repeated over and over again per game:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
 
[Event "Valencia"]
[Site "Valencia"]
[Date "1475.??.??"]
[Round "?"]
[White "De Castellvi, Francisco"]
[Black "Vinoles, Narcisco"]
[Result "1-0"]
[ECO "B01"]
[PlyCount "41"]
[EventDate "1475.??.??"]
[EventType "game"]
[EventCountry "ESP"]
[Source "ChessBase"]
[SourceDate "2007.11.25"]

1. e4 d5 2. exd5 Qxd5 3. Nc3 Qd8 4. Bc4 Nf6 5. Nf3 Bg4 6. h3 Bxf3 7. Qxf3 e6 8.
Qxb7 Nbd7 9. Nb5 Rc8 10. Nxa7 Nb6 11. Nxc8 Nxc8 12. d4 Nd6 13. Bb5+ Nxb5 14.
Qxb5+ Nd7 15. d5 exd5 16. Be3 Bd6 17. Rd1 Qf6 18. Rxd5 Qg6 19. Bf4 Bxf4 20.
Qxd7+ Kf8 21. Qd8# 1-0


so i made this code that works, at least it works for the first chunk(1,5GB)
at the second chunk it crashes around 26mb of output(at about 13%)
Also the 3rd chunk somehow crashes, but then at 49,9%(105mb of output approx.). i tried to rerun the program, but it always crashes at the same point.

the program reads line by line using the getline function.

it makes strings, and everytime you find a piece of information you put it in a string, and when you find ''[event '' it outputs all strings.(if no information was found it gives NOval.

Do you guys have any idea why the program could crash? i rename the chunks everytime to Allespgn.pgn and then store the output (does that matter?)

greetings,
Roland
Last edited on
poiioppoiiop (9)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
 
#include <string>
#include <iostream>
#include <fstream>
using namespace std;
#include <cstdio>
#include <algorithm>
int main()
    {
                std::string a1 = "";
                std::string a2 = "";
                std::string a3 = "";
                std::string a4 = "";
                std::string a5 = "";
                std::string a6 = "";
                std::string a7 = "";
                std::string a8 = "";
                std::string a9 = "";
                std::string a10 = "";
                std::string a11 = "";
                std::string a12 = "";
                std::string a13 = "";
                std::string a14 = "";
                //std::string a15 = "";
                std::string a16 = "";
                std::string a17 = "";
                std::string a18 = "";
                std::string search_str = "[" ;
                std::string search_str3 = "[Event " ;
                std::string search_str2 = "\"]" ;
                ofstream out("fileforexcel.txt");
                std::ifstream file("Allespgn.pgn" ) ;
                std::string line ;
                int x = 0; //(date, only year or not only year)
                int y = 0; //(eventdate)
                bool namen = true; // false for no colomn names
               
                if (namen == true)
                {
                    out<<"Event\tSite\tDate\tRound\tWhite\tBlack\tResult\tECO\tPlyCount\tEventDate\tEventType\tEventrounds\tEventCountry\tSource\tSourceDate";
                }
Last edited on
poiioppoiiop (9)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
 
                while( std::getline( file, line ) )
                {
                    if( line.find(search_str3) != std::string::npos)
                    {
                        out<<a1<<a2<<a3<<a4<<a5<<a6<<a7<<a8<<a9<<a10<<a11<<a12<<a13<<a14<<a16<<a17<<a18<<"\n";
                        a1 = "NOval\t";
                        a2 = "NOval\t";
                        a3 = "NOval\t";
                        a4 = "NOval\t";
                        a5 = "NOval\t";
                        a6 = "NOval\t";
                        a7 = "NOval\t";
                        a8 = "NOval\t";
                        a9 = "NOval\t";
                        a10 = "NOval\t";
                        a11 = "NOval\t";
                        a12 = "NOval\t";
                        a13 = "NOval\t";
                        a14 = "NOval\t";

                        a16 = "NOval\t";
                        a17 = "NOval\t";
                        a18 = "NOval";
                        }

        if( line.find(search_str) != std::string::npos && line.find(search_str2) != std::string::npos) //vind kaartnaam
                {
                  // out<<line<<"\t";
                  //out<<Kolomnummer<<"\t";
                    if(line.find(search_str3)!= std::string::npos)
                       {
                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a1 =line+"\t";
                       }

                    if(line.find("[Site ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a2 =line+"\t";
                       }
                    if(line.find("[Date")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2-6*x, line.end());
                        a3 =line+"\t";
                       }
                    if(line.find("[Round")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a4 =line+"\t";
                       }
                    if(line.find("[White ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a5 =line+"\t";
                       }
                    if(line.find("[Black ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a6 =line+"\t";
                       }
                    if(line.find("[Result ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a7 =line+"\t";
                       }

                    if(line.find("[ECO ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a8 =line+"\t";
                       }
                    if(line.find("[WhiteElo ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a9 =line+"\t";
                       }
                       if(line.find("[BlackElo ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a10 =line+"\t";
                       }
                    if(line.find("[PlyCount ")!= std::string::npos)
                       {


                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a11 =line+"\t";
                       }
                    if(line.find("[EventDate ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2-6*y, line.end());
                        a12 =line+"\t";
                       }
                    if(line.find("[EventType ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a13 =line+"\t";
                       }
                    if(line.find("[EventRounds ")!= std::string::npos)
                       {


                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a14 =line+"\t";
                       }
                    if(line.find("[EventCountry ")!= std::string::npos)
                       {


                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a16 =line+"\t";
                       }
                    if(line.find("[Source ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2, line.end());
                        a17 =line+"\t";
                       }
                    if(line.find("[SourceDate ")!= std::string::npos)
                       {

                        unsigned found = line.find(" ");
                        line = line.substr(found+2);
                        line.erase (line.end()-2-6*x, line.end());
                        a18 =line+"\t";
                       }


                }
            }
            file.close();
            out.close();
    }
iQChange (374)
Come on; where's the source?
poiioppoiiop (9)
srry was still posting the sourcecode, but there is a 8000 token limit. took a while, the database is btw chessbase/megabase
poiioppoiiop (9)
it crashed around :
$16 Qd8 29. Bg4 $1 Bxg4 $2 (29... Qc8 30. hxg6+ fxg6 31. Qh2+ Kg8 32. Rxf8+
Kxf8 $16) 30. Qxg4 Qd6 31. Bg5 {/\Kg2,Rh1
} gxh5 32. Qxh5+ Kg8 33. Kg2 f6 (
33... Qg6 34. Qxg6 fxg6 35. d6 $18) 34. Be3 Rf7 35. Qf5 b4 36. Ra1 (36. Qc8
poiioppoiiop (9)
ok i found some more information. i am using Vim (some text editor for really big files, and everywhere there is a "^Z", the program crashes but i am unable to find out how to remove them, or finding out what it means.
does anybody know?
poiioppoiiop (9)
nvm i found it, there is a --> sign in the textfile that kills the getline loop.
Topic archived. No new replies allowed.