Create Big Multi-d Array w/o Causing Memory Issue

[nsi]

Dear C++ programmers!

I have been trying to create a big multi-d array but each and every time there appears memory issues. Here is what I have tried.

(1) Using "new" to generate a dynamic array:

using namespace std;

int main()
{
int imax = 1000;
int jmax = 1000;
int kmax = 1000;

double ***U = new double **[imax];
for (int i = 0; i < imax; i++) {
U[i] = new double *[jmax];
for (int j = 0; j < jmax; j++) {
U[i][j] = new double[kmax];
}
}

return 0;
}

When I compile and run it, MS Visual Studio says:

Unhandled exception at 0x75D2C41F in xxx.exe: Microsoft C++ exception:
std::bad_alloc at memory location 0x004EFD7C.

(2) Using "vector" to generate a dynamic array:

using namespace std;

int main()
{
int imax = 1000;
int jmax = 1000;
int kmax = 1000;

vector<vector<vector<double>>> U(imax);
for (int i = 0; i < imax; i++) {
U[i].resize(jmax);
for (int j = 0; j < jmax; j++) {
U[i][j].resize(kmax);
}
}

return 0;
}

Same thing happened;

(3) Generating a static array variable.

static double U[1000][1000][1000];

This time the systems warns that the size is too large.

I have to admit that those are all my knowledge of creating a multi-d array in C++. Any idea of different ways to create an array of this size w/o causing memory issues will be much appreciated!

Best,

N. Si

[mbozzi]

How much memory does your computer have?

You obviously need more than 8GB free.

[nsi]

It's a 16GB.

[mbozzi]

What's your compiler's target?

You need to be compiling 64-bit code on a 64-bit system.

[dhayden]

That's a really REALLY big array. What are you using it for? Maybe there's a more efficient data structure.

To elaborate on mbozzi's points, you're allocating a 1000x1000x1000 array so there are 1 billion elements. Since each element is a double and double's are usually 8 bytes, you need at least 8GB of RAM (actually just 8GB of address space), assuming that your OS handles virtual memory).

32 bits can only address 4GB of memory so you need a 64 bit program, which requires a 64 bit OS.

But to reiterate my original point, what are you using this for? There's a very good chance that you can solve your problem without such a huge array.

[zaphraud]

The problem is you're using Microsoft Visual Studio.

Install cygwin. It's free. The setup isn't the friendliest, but it's not terribly difficult if all you want to do is run command line stuff..

Your code compiled just fine for me.

Also crashed Firefox when I tried to reply, because there was no memory left on this laptop - 32GB of flash and 8GB of ram, total. But hey, 10 hours of battery life and super light. Guess I need to fill the empty SSD bay if I want to run your code...

But at least it didn't just up and die at a 4GB limit... It started slaughtering everything else hoping it had a fighting chance. :-)

[jonnin]

There is nothing wrong with visual studio. I have used it on large problems for years. You may need to flip some compiler flag settings, but you do that on g++ also. Cygwin is also nice, but its a royal pain to make gui programs or programs you can give to other window's users (need a lot of large dlls) with it.

-It is usually better to not put all that in memory at once, even if it fits, though. Consider working on it in bits. We need to know more to recommend a way to break it up.

right, static arrays can't be anywhere near this size.
vector and new do effectively the same thing. vector is better.