At the moment I'm importing the discrete Laplacian into my matrix (n x n, square matrix). It has no more than 5*n non-zero elements. When n > 20000, I get a segmentation fault, i.e. at about 100,000 elements.
Buying more RAM? Although that doesn't really sound like it should cause a problem.
Just before the segfault, what's the size of the biggest inner map?