Saving values from a .dat file into a matrix

[fabstr1]

How do I write a program that saves values from a .dat file into an array?

I have a .dat file with 6 rows and more than 10000 columns. I want to store the columns separately in an array, so that I can plot the array against another array later on. How do I do that, I'm beginner at C++.

  #include <iostream>
#include <string>
#include <stdio.h>
#include <fstream>
using namespace std;




void Test(){

	double Bx[10000];
	
	short loop = 0;
	
	string line;
	ifstream myfile ("Test.dat");
 	if (myfile.is_open()){
 		while (!myfile.eof())
 		{
 			getline(myfile,line);
 			Bx[loop] = line;
 			cout << Bx[loop] << endl;
 			loop++;	
 	
 		}
 	myfile.close();
	}
	else cout << "Unable to open file";
 
    	system("PAUSE");
  
    	}

[Ganado]

First, let's focus on just reading from the file. What does the file look like? Give us an example of some of the rows/columns.

If you're not sure what the maximum size of the largest dimension of your data will be, you may want to consider using a vector instead of a hard-coded array (and an array of size 10000 certainly won't be enough to hold "more than 10000 columns").

Do you need to store the entire set of data (6x10000+ items), or is only a subset of the data in each column needed?

PS: Usually, the term "save" would mean "write to a more-permanent type of storage, from the less-permanent storage", and you'd say "load" if you're loading stuff from a file into memory, but it's not too important.

[Handy Andy]

Hello fabstr1,

Some things not mentioned yet:

The C header file "<stdio.h>" should be "<cstdio>" for a C++ program. Even with that change you do not need it. The "<iostream>" covers everything that you need and it is possible that "iostream" may include "cstdio".

Your function defines: double Bx[10000];, but you say that there 6 lines of 10000+ columns. Not quite enough for a 1D array. Your description would suggest a 2D array. Next problem you may find is 10000 * 6 * 8 = 480000 bytes. You may run out of memory just trying to define the array. NO sure, but something to think about.

Next is the array is defined in the function, so when the function ends so does that array when it is destroyed along with any other variable(s) you define.

You define: short loop = 0;, but do you know what the largest number a short can hold. When I check it on my computer it says that a short is 2 bytes and a signed short can store a number between ( -32,768 to 32,767). That is a long way from 10000+.

if (myfile.is_open()). Checks if the file is open, but it may be open and not usable. What you should do is something like this:

if (!myfile)
{
    std::cerr << "\n File \"" << "Test.dat" << "\" did not open.";

    return 1;
}

This way you can continue your program with out the if statement.

Your while loop will does not work the way that you are thinking. You are checking for "eof" before you read the file which might set the "eof" bit, then you will try to store a number at element 10001 which does not exist in your array.

Line 21 reads an entire line of the file and stores it as a "string".

Line 22 is trying to store a "std::string" into 1 element of the array that is defined as a double. This will not work.

And even if you would get to line 23 you will print an element of the array that contains a garbage value because the array was never initialized or receiver a proper value.

Loop will eventually roll over to a negative number when it exceeds (32,767) and you can not use a negative number to access an array. Even an unsigned short would go to ( 65,535).

Running most of "main" on an if/else may work, but it is not the best way to write a program. The if statement I showed you would come after line 17 and if it is true you would leave the program to deal with the problem. This removes access to the rest of the program that you do not need to continue with.

system("PAUSE"); works and I believe that lower case letters are what you need, but it may not be case sensitive.

I like using this in its place:

// A fair C++ replacement for "system("pause")". Or a way to pause the program.
// The next line may not be needed. If you have to press enter to see the prompt it is not needed.
std::cin.ignore(std::numeric_limits<std::streamsize>::max(), '\n');  // <--- Requires header file <limits>.
std::cout << "\n\n Press Enter to continue: ";
std::cin.get();

return 0;  // <--- Not required, but makes a good break point. 

This is what I see for now.

Andy