Text file loader bytes to readable chars

[rob1609]

Hi
I wrote long time ago a CSV file loader which basically reads each line in text file and splits the string into columns (smaller strings) on each occurence of ',' for example my text file looks like this:


Bar code,Product Name,Price,Quantity
1230439483,Strawberry jam,1.20,12


The most important of code in this function looks like:

bool CFileIOManip::readAllRows( CStdioFile *file )
{
CString row;
int numCols, lineNumb = 0;

//read txt rows until the end of file
while (file->ReadString( row ) != NULL ) // gets string from file
{
if(!row.IsEmpty())
{
std::vector<CBuffer> vecColumns;
getAllColumnsInRow(const CString &row, std::vector<CBuffer> &vecColumns,Ok); // takes string 'row' returns columns in 'vecColumns'
}
}
}

the above function will split each string from CSV file into 5 columns and store it in a vector container for further processing.
it basically gets each character one by one and looks for ',' like:

CString str;
for(int i....)
CString c=row.GetAt(i);
if(!comma)
str+=c;


It works great but I have one problem with carriage returns.
When a string (line in file) contains in the middle a hidden carriage return my program will read this string up to the carriage return and go to next line and read the remainder of the string as new record (line) which is wrong.
For example this one record is on 2 lines instead of 1 line:

(in Notepad I see also this)

1230439483,Strawberry jam,
1.20,12

instead of this:

1230439483,Strawberry jam,1.20,12

my function will see this as 2 records as there are two lines because carriage return is also in this line between Straberry Jam and 1.20

I read that with CString it cant be done and somebody suggested that I should write a new loader which will load bytes instead of strings and then filter out all carriage returns and this way I could read the whole line regardless if it has a carriage return or not but then how do I know when there is an end of line and when to proceed to next line?

I am starting to experiment with ifstream to read bytes like:

ifstream inFile.open( "test.txt", ios::in|ios::binary|ios::ate );
char* data = 0;
data = new char[ size+1 ]; // for the '\0'
inFile.read( data, size );
data[size] = '\0' ; // set '\0'
// print data
for ( size_t i = 0; i < strlen(data); i++ )
{
cout << data[i];
}