reading in multidimensional array of unknown size

[zolo]

Hi,

I am back to the forum after a long period and I have probably an easy question but somehow I could find the answer yet. I would like to read in a two dimensional array of unknown size from an ascii (text file) file.
The following code I found on cplusplus website and works like charm meaning that If I print FILE out to the terminal I get back the array but how do I transform it to double? In principle it does not matter whether it is a matrix (array[][]) or vector (array[]) I just want it in double not char.

 ifstream file_in;
file_in.open(argv[2]);
  if (file_in) {
    // get length of file:
    file_in.seekg (0, file.end);
    int length = file_in.tellg(); //determining length 
    file_in.seekg (0, file_in.beg);
     cout << length << "\n";
  
    char*  FILE = new char [length];
   
    file_in.read (FILE,length);

    file_in.close();


  }

Is there any way to determine how many doubles are in the file not the number of characters?

Thanks

[keskiverto]

a two dimensional array of unknown size

You must know something. Let say that the file contains numbers 1, 2, 3, 4, 5, 6, 7, 8.
It could be 1D array with 8 elements, or 2x4 or 4x2, or even a 2x2x2 3D array.

In other words, something has to reveal (all but) one of the dimensions.

std::vector<double> array;
double value = 0.0;
file_in << value;
array.push_back( value );

[zolo]

Thanks for the prompt reply. I figured out that I only have doubles in the txt file so I could calculate the number of rows in the file giving, me one dimension. The other dimension I can set in an other script so actually the array has now a known size now! :). Although I still cannot read in my data I get an error message:

invalid types ‘double[int]’ for array subscript

Where do I go wrong?

ifstream file_in;
file_in.open(argv[2]);
  if (file_in) {
    int number_of_lines = 0;
    string line;

    while (getline(file_in, line))
         ++number_of_lines;
        cout << "Number of lines in text file: " << number_of_lines << "\n";
        double FILE_IN[number_of_lines][2];
     for (int i=0; i<number_of_lines; i++){
         for (int j=0; j<2; j++){
     file_in >> FILE_IN[i][j];
         }
     }

    file_in.close();

[keskiverto]

First, memory for array FILE_IN would be reserved from stack. The size of the memory should be known during compile-time. In your program the size is revealed during run-time. Latest C standard does support VLA (variable length arrays) in order to allow run-time size determination, and next update of C++ standard is likely to have it too, despite some opposition.


So, you have now revealed that the file contains double numbers. That there are N lines and each line contains about M values? That the endlines tell about array dimensions?

What to do?

Make a 2D array std::vector< std::vector<double> > array;
A: Read a line as std::string
Create a std::istringstream object X from the line
Create temporary 1D array for row std::vector<double> row;
While X: read a double from the stream X and append to the row
Append row to array
Repeat from A.

[zolo]

Hi,

Thanks for the advice I am trying to follow what you wrote but I still have some trouble with the string to double conversion. This code works but I dont know how to define a dynamic string vector does

new

work with strings? The actually array is doubles[17254][2]. The second dimension is known

string line
vector< string > lines( 32000 );
    int i = 0;
    while(getline( file_in, line ) ){
        lines[i] = line;
        ++i;
    } 

I do not want to define the length of the vector because it can be any size even bigger than 32000!

This didnt work:

string line
vector< string > lines( 32000 );
    int i = 0;
    while(getline( file_in, line ) ){
        lines[i] = atof(line);
        ++i;
    } 

[keskiverto]

The second dimension is known

Why did you not tell that in the first place?

You should not say that array contains 32000 items, if it won't. That is why you should append rather than assign to vector.

Where did that vector<string> came from? You were working with doubles, weren't you?

You want 2D array and the second dimension is two? std::vector<std::array<double,2>> array;
You guestimate that array will contain about 12345 rows? array.reserve(12345);
You have only doubles in a file?

std::array<double,2> row;
while ( file_in >> row[0] >> row[1] )
{
  // append row to array here.
}

[zolo]

Hi

Since I am kind of lost right now with the input output part of c++ the string came from nowhere I am just trying out everything I can!
Yes I only have doubles in the file but the number of rows can be different from file to file. They are produced by another script. The only thing I could fix in the other file is that the number of columns is maximum 2. So every file has 2 columns.

thanks!

[zolo]

std::array<double, 2> row;

this gives me the error the array is not a member of std. I did include <array> header.

[Smac89]

Post what a sample input file looks like

[zolo]

1 35.35
2 45.33
6 87.43
12 2345.345
... ... etc.

It is a very simple file with only two columns and only numbers separated by spaces. the only difficulty is that the number of rows is unknown.

[Smac89]

size_t row_count(std::istream &file) {
	file.seekg(std::ios_base::beg);
	size_t count = 0;
	int ch;
	for (ch = file.get(); ch != -1; ch = file.get())
	{
		while(ch != '\n')
			ch = file.get();
		count++;
	}
	
	file.seekg(std::ios_base::beg);
	return count;
}

Or:

size_t row_count(std::istream &file) {
	file.seekg(std::ios_base::beg);
	size_t count = 0;
	std::string buf;
	
	while(std::getline(file, buf))
		count++;
	
	file.seekg(std::ios_base::beg);
	return count;
}

usage:

int main()
{
	std::ifstream myfile("sample.txt");
	size_t num_rows = row_count(myfile);
	...
	return 0;
}

[JLBorges]

> this gives me the error the array is not a member of std. I did include <array> header.

Compile with the -std=c++11 option.

#include <iostream>
#include <fstream>
#include <vector>
#include <array>

std::vector< std::array<double,2> > create_from_file( const char* file_name )
{
    std::vector< std::array<double,2> > array_2d ;
    std::ifstream file( file_name ) ;

    double a, b ;
    while( file >> a >> b ) array_2d.push_back( {a,b} ) ;

    return array_2d ;
}

int main()
{
    auto array_2d = create_from_file( "my_data.txt" ) ;

    std::cout << array_2d.size() << " rows were read\n" ;

    if( array_2d.size() > 5 )
        std::cout << "row 5: " << array_2d[5][0] << ' ' << array_2d[5][1] << '\n' ;
}

[zolo]

Is it possible without c++ 11?

[JLBorges]

Yes.

#include <iostream>
#include <fstream>
#include <vector>
#include <cassert>

struct row
{
    double a[2] ;

    double& operator[] ( std::size_t pos ) { assert(pos<2) ; return a[pos] ; }
    double operator[] ( std::size_t pos ) const { assert(pos<2) ; return a[pos] ; }
};

std::vector<row> create_from_file( const char* file_name )
{
    std::vector<row> array_2d ;
    std::ifstream file( file_name ) ;

    row r ;
    while( file >> r[0] >> r[1] ) array_2d.push_back(r) ;

    return array_2d ;
}

int main()
{
    std::vector<row> array_2d = create_from_file( "my_data.txt" ) ;

    std::cout << array_2d.size() << " rows were read\n" ;

    if( array_2d.size() > 5 )
        std::cout << "row 5: " << array_2d[5][0] << ' ' << array_2d[5][1] << '\n' ;
}

[zolo]

Thanks!
Now, just out of curiosity, lets assume that we live in a perfect world where every cow is spherical. Lets assume that I know everything about my array in the ASCII. It is only one column vector and contains only 100 doubles. Could this work:

double data[99];
ifstream file_in(argv[1]);

file_in.open();
for (int i=0; i<99; i++){

file_in >> data[i];
}

file_in.close(); 

[JLBorges]

> Lets assume that I know everything about my array in the ASCII.
> It is only one column vector and contains only 100 doubles.
> Could this work:

Yes, it would work. Though it would still be bad code; code that assumes that nothing can ever change over a period of time.

Writing code as if we do not know everything (even when we do know everything) is a good idea; abstraction is just another name for deliberate, selective, ignorance.

This would work today, and it would also work tomorrow.

if( argc > 1 )
{
    std::vector<double> data ;
    std::ifstream file( argv[1] ) ;
    double d ;
    while( file >> d ) data.push_back(d) ;
    
    // ....
}

[giblit]

contains only 100 doubles

double data[99];
for (int i=0; i<99; i++){

You only have 99 anyways.

[zolo]

Thanks for the solutions I am trying out out everything since to this is the ninja bit of c++ reading in files! It seems I have a lot to go until I understand the full story. :)