General advice for handling (possibly much) data

[red vintage]

Hi there!

This is my first post in a programming forum and my first post in English, too. I will try to express my problem as clear as I possibly can.

I am implementing a numerical algorithm and - after changing some parameters - my program interrupted due to a segmentation fault.
Then I isolated the problem and discovered that one cannot declare such a (big) array:

  double points[262144];

Now the algorithm - from the highest point of view - just changes such an array in several steps, the number of steps is fixed in advance. (In the first step, it is filled with some input values.)
In each step, there's a loop over all elements. Depending on the values of these elements, new values a written to a temporary array.
Then the temporary array is copied into the original one, and a new step starts.

Now - I think - I have three major problems:

1.
I cannot handle this much data via this array, but in a way I have to. It might be interesting that the size of the array which I fix in the beginning is the worst case for the algorithm. Possibly I don't need that much space, but I don't know in advance. It depends on the problem itself.
2.
The runtime is crucial here. Is there a way I can save the copy-procedure in each step of the algorithm?
3.
In each step, the data has to be passed to several sub-functions. (Using the array, I can comfortably and efficiently pass the pointer of the first element to the sub-functions so that I have access to the data.)

Is there a 'data structure' (in terms of something similar to 'my' array) which can solve the problem 1. and 3. (2. should be optional in the beginning)?


I would be very grateful for some advice. Thanks in advance!
red vintage

[Chervil]

Probably the simplest and safest way is to use a std::vector

    std::vector<double> points(262144);

The elements of the vector can be accessed using the same syntax as an array such as points[0] for the first element and do on.

The memory allocated will be automatically released when the vector goes out of scope e.g. at the end of main().

A vector can also be also be resized during program execution if necessary, which might be useful in your case.

http://www.cplusplus.com/reference/vector/vector/

An alternative is to use new [] and delete [] which is ok too, but tends to need more care.

http://www.cplusplus.com/doc/tutorial/dynamic/

new values a written to a temporary array.
Then the temporary array is copied into the original one,

You might avoid the need for copying if a pointer to the two arrays are used and simply swap the values of the pointers. A vector makes this simple as there is a built-in swap function:
http://www.cplusplus.com/reference/vector/vector/swap/

If you want to pass a large object such as a vector to another function, then pass it by reference.

[red vintage]

Thanks for your answer, Chervil!

Actually when I searched for topics concerning this issue in the forum, I read about the class "vector" (please correct if I'm using the terms "class", "object" and so one in a wrong way).

But I have been to narrow-minded.
I thought it can't be possible to have a pointer for a variable of vector-type, because I thought such a variable is "more" than an array ... and therefore didn't give it a chance.

But a function declaration such as

void subfunction(std::vector<double> &points);

works perfectly.

So how can I imagine a variable defined as follows:

	vector<double> points(100);

?
(Intuitively it's "more" than an array, but the function above shows that it's some "normal" space in the memory that i can adress via pointers...)


The swap-function puzzles me a bit (but I am very grateful for it, though).
Whenever I tried to swap the names of my array, the original and the temporary one, I had the problem that an array is not a pointer.
So when I had two arrays, the only way to swap them was:

double points[];
double points_temp[];

double *first = points;
double *second = points_temp;

double *temp = first;
first = second;
second = temp;

But then the names have changed into "first" and "second", which was not okay because of the loop.
So does anyone know how the swap-function does it?

[Chervil]

Well, a vector is "more than an array" because it keeps track of not just the array contents, but also such things as the size and capacity. But when swapping two vectors, it can simply exchange one or more internal pointers. This means its possible to swap two vectors of the same type, even if their sizes are different.