Part of the files cannot be reached under the same folder

[benhsu]

Hi all,

I have a few files in CSV format under a folder, I tried to combine these CSV files as one. I applied a for loop to process the files one by one; however, part of the files cannot be reached. All of the files were produced by the same program, so i think that there is no encoding issue.
Can anyone help me? I want to figure out what's wrong. My source codes are in below.

thanks in advance.

#include <sys/types.h>
#include <sys/stat.h>
#include <dirent.h>
#include <errno.h>
#include <vector>
#include <iostream>
#include <fstream> /* ifstream class */
#include <string> /* string class */
#include <cstdlib> /* atoi */
#include <sstream> /* istringstream class */
#include <string.h>


using namespace std; 

#define filename1 "1dtrainingdata.csv"


int getdir(string dir, vector<string> &files);
int main()
{ 
	struct stat buf;
	string path = "d:\\GC_Vectors";
	int i,j,ROW,COL,ROW_1;
	string **arr = NULL;
	string filename;
	
	fstream fin;
	fstream fin_1;
	ofstream fp1(filename1); 
	
	string dir = string("d:\\GC_Vectors");
   
    
    vector<string> files = vector<string>();
    getdir(dir, files);    
 
    
    	
	for(i=0; i<files.size(); i++)
	{ 
   
    ROW=0;
    COL=0;
    ROW_1=0;
    filename = files[i];
    cout << "stage 1 " << filename << endl;                          
	        
	
	if(stat(filename.c_str(), &buf) != -1)
	{
	
		if (filename == "."||filename == "..")
		{
			continue;
		}
		else
		{
			fstream fin(filename.c_str(),ios::in);
			cout << "stage 2 open " << filename <<endl; 
			
			string line;
			int linecount=0;
	
		while (getline (fin, line, '\n'))  
		{
		linecount++;
		
		istringstream stream(line);
		string item;
		int itemcount=0;
									
			while (getline (stream, item, ',')) 
			{		
					itemcount++;					
			}						
				ROW = itemcount;															
		}	
		COL = linecount;
		cout << COL << " " << ROW << endl;
		system("pause");
		
		fin.close();
		fin.clear(); 
	

	fstream fin_1(filename.c_str(), ios::in);
	
	string outputstring, string_1;
	arr = new string*[COL];
	
	for (i=0; i<COL; i++)
	{
		arr[i] = new string [ROW];		
	}
	
	for(i=0; i<COL; i++)
	{
		getline (fin_1, string_1, '\n');
		istringstream stream(string_1);
		
		for(j=0; j<ROW; j++)
		{		
			getline (stream, outputstring, ',');				
			arr[i][j] = outputstring; 
									
		}	
		
		
	}			
	
	fin_1.close();
	fin_1.clear();

		
	ROW_1 = ROW - 5;
	

    if(!fp1)
	{
        cout<< "stage 4 Fail to open file: " << endl;        
    }
    else
    {
    	for (i=0; i<COL; i++)
    	{
    		for(j=0; j<ROW_1; j++)
    		{
    			fp1 << arr[i][j] << ",";
    					
			}
			fp1 << endl;
		}		
	}
		
	}	
	delete [] arr;		
	
}
else
{
	cout << "file is unable to open " << filename <<endl;
	system("pause");
	continue;
}
}
	
	fp1.close();
	fp1.clear(); 
	
	return 0;
}



int getdir(string dir, vector<string> &files){
    DIR *dp;
    struct dirent *dirp;
    if((dp = opendir(dir.c_str())) == NULL)
	{
        cout << "Error(" << errno << ") opening " << dir << endl;
        return errno;
    }
    while((dirp = readdir(dp)) != NULL) 
	{
        files.push_back(string(dirp->d_name));
    }
    closedir(dp);
    return 0;
}

[Computergeek01]

Why are you making this so complicated? The CSV format is just plain text, it literally stands for "Comma Separated Value". Just read from the source file and append to the destination. Also, please use code tags, as you can see not doing so strips out your white-space and as a result makes your post unnecessarily difficult to read.

[Chervil]

however, part of the files cannot be reached.

What cannot be reached? Some of the actual files, or the data inside the files? Is it that the last five items on each line are missing.

[benhsu]

Hi Chervil, thanks for your reply.
I am unable to reach part of the actual files.

I tried to figure out what's wrong by applying "cerr << "Error: " << strerror(errno);", and got a message "no such file or directory" for those files cannot be opened.
This message makes me confused; any idea for this?

[benhsu]

Hi Computergeek01.
I am not familiar with this interface, could you guide me how to use code tags?
thanks in advance

[Chervil]

I tried to figure out what's wrong by applying "cerr << "Error: " << strerror(errno);", and got a message "no such file or directory" for those files cannot be opened.

I'm not sure what it means either. Maybe there are some other types of files (subdirectories, hidden files, etc.) in the given directory. I tried a much-simplified version of your code with no problems, but my folder contained just three files of type .csv and nothing else.

Maybe this example might help, though it doesn't do exactly what you want,
http://www.cplusplus.com/forum/beginner/10292/#msg47965

[Chervil]

How to use code tags: http://www.cplusplus.com/articles/jEywvCM9/

[Computergeek01]

@ OP: It looks like you're trying to use Linux libraries in a Windows environment. Are you running this in Cygwin?

[benhsu]

Hi Computergeek01, no, I run it on Windows 10.

[benhsu]

How to use code tags: http://www.cplusplus.com/articles/jEywvCM9/

Hi Chervil, thanks it work.

I think that I figure out what's wrong in my codes.
I open the input file twice through fstream fin and fin_1; it runs correctly in stage one (fstream fin) but stage two (fstream fin_1).

any ideas for this?
thanks

fstream fin_1(filename.c_str(), ios::in);
	
	string outputstring, string_1;
	arr = new string*[COL];
	
	for (i=0; i<COL; i++)
	{
		arr[i] = new string [ROW];		
	}
	
	for(i=0; i<COL; i++)
	{
		getline (fin_1, string_1, '\n');
		istringstream stream(string_1);
		
		for(j=0; j<ROW; j++)
		{		
			getline (stream, outputstring, ',');				
			arr[i][j] = outputstring; 
									
		}	
		
		
	}			
	
	fin_1.close();
	fin_1.clear();