Stuck in Memory Allocation

[aps83]

Hi,

This problem has been bugging me all day. It seems that when the size of the memory allocation is over a certain size the program just freezes, rather than returns an exception.

This can be observed by changing the numNodes variable.

The correct operation should simply be that when the allocation fails the size of the allocation is reduced and attempted again until successful. This happens for me up till around numNodes = 39000000;. At this point the initial allocation never completes or returns an error.

Any suggestions?

Windows 7 64-Bit. MS VS 2008 Pro.

#include <iostream>
#include <math.h>

using namespace std;

typedef struct {
   float* pc[2];   
} nbr;  //neighbour past/current pressure pointers

class node
{
public:
	
	int* nid;
	int* Nbrs[6];
	int* nodeType;
	int* xyz[3];		
	int *bid; 	
	float* pc[2]; 
	nbr Nbrs_pc[6]; 		
	node::node(void)
	{
	}
	~node(void)
	{
	}
private:
	float* bb[2];	
};

int main(int, char *[])
{
	
	node *tmpNodeBlk = NULL;
	int block_size = 0;
	int tmpblksize = 0;
	int blockcnt = 0;
	int num_blocks = 0;
	int rem_size = 0;
	int numNodes = 40000000;//39500587;
	
	block_size = numNodes;
	blockcnt = block_size;
	num_blocks = 1;

	bool allocated = false; 

	while ( allocated == false )
	{
		allocated = true;
		try	
		{ 
			cout << "Trying to Allocate Memory... " << endl;
			tmpNodeBlk = new node [block_size];
		}
		
		catch (std::bad_alloc ba)
		{ 
			cout << "Allocation Failed, Reducing Block Size... " << endl;
			block_size = (int)floor(block_size/2.0);
			blockcnt = block_size;
			num_blocks = num_blocks*2;
			rem_size = numNodes-(num_blocks*block_size);
			allocated = false;
		}
	}
	
	cout << "Allocation Completed" << endl;
	cout << "Number of Blocks: " << num_blocks << endl;
	cout << "Block Size: " << blockcnt << endl;


	system("PAUSE");

	delete []tmpNodeBlk;
 
  return EXIT_SUCCESS;
}

[simeonz]

It's a stab in the dark for me, but have you tried catching by reference (i.e. catch (std::bad_alloc& ba)).

Regards

EDIT: Also, check out the size of node (with sizeof) at the beginning of the program. I mean, with field alignment it may turn out more than 100 bytes. You are allocating 40,000,000 objects, so the allocator may have to allocate more than 4G and that may be what bugs him. Say, it may have a wrap-around problem with its internal calculations.

[aps83]

Thanks for your reply.

The catch (std::bad_alloc& ba) made no difference.

The sizeof(node) turns out to be 112 bytes.

Is there any elegant way to detect and avoid this issue? Rather than arbitrarily asking

if (block_size > 30000000) 
         { REDUCE BLOCK SIZE UNTIL < 3000000 } 

Thanks

[simeonz]

Actually, I don't think there is something wrong with your code. I think this is bug with the allocator:

Standard 03 wrote:

[Note: unless an allocation function is declared with an empty exception-specification (15.4), throw(), it
indicates failure to allocate storage by throwing a bad_alloc exception (clause 15, 18.4.2.1); it returns a
non-null pointer otherwise. If the allocation function is declared with an empty exception-specification,
throw(), it returns null to indicate failure to allocate storage and a non-null pointer otherwise. ] If the
allocation function returns null, initialization shall not be done, the deallocation function shall not be called,
and the value of the new-expression shall be null.

That said, you can check smth like:

if (block_size * (sizeof(node) + some_space) > std::numeric_limits<std::size_t>::max())
{
  block_size = std::numeric_limits<std::size_t>::max() / (sizeof(node) + some_space);
}

http://www.cplusplus.com/reference/std/limits/numeric_limits/

You can not compute the memory actually allocated precisely, because there is some service memory that is prepended to the chunk that is returned to you. This is clarified here:

Standard 03 wrote:

A new-expression passes the amount of space requested to the allocation function as the first argument of
type std::size_t. That argument shall be no less than the size of the object being created; it may be
greater than the size of the object being created only if the object is an array. For arrays of char and
unsigned char, the difference between the result of the new-expression and the address returned by the
allocation function shall be an integral multiple of the most stringent alignment requirement (3.9) of any
object type whose size is no greater than the size of the array being created. [Note: Because allocation
functions are assumed to return pointers to storage that is appropriately aligned for objects of any type, this
constraint on array allocation overhead permits the common idiom of allocating character arrays into which
objects of other types will later be placed.]

This is why you need to guess/adjust the some_space value from the code above accordingly. It is usually a small fixed amount, although hypothetically the standard allows it to vary from allocation to allocation.

Regards