How to open a large text file in a TextBox?

[tango2k]

Hi all, I'm new here and I would like you to help me out with something.
Turns out I need to open a large (6 mb) text file in a Windows Form (using C++). I've thought of doing this with a TextBox or a RichTextBox (slighty faster) but they all take more than 10 seconds to open while memory usage jumps from 20 mb to 450mb!!!!
My system is a quad-core 2.5 ghz and I have 4 gigs of RAM. The program is supposed to be used in slower systems (i'm thinking of 1.8 ghz P4 with 1 gig of RAM).
Is there any efficient implementation of textboxs available to use? Shall I implement my own version? If so, do you have any recommendations?
Thanks in advance!
Rick

[johnkravetzki]

That's an interesting question. I just tested a large txt file (18MB) - it takes a while to open it, while the CPU Usage jumps nearly to 50% (Win XP, Dual Core 2GHz).

Anyway, I'm sure it's possible when using C++. You read the entire file into memory and parse it. No idea how could you make it in C++/CLI (streamreader?)

I'm not sure about multithreading - what I mean here is to read the file by part, each part in different thread - but will it be more efficient?

[tango2k]

Thanks for your answer John!!
I'm using a streamreader (somebody suggested me to use a raw streamreader and then convert it into a text stream in order to make is faster - will see about that).
Perhaps using multithreading could shorten load times (if run in a multi-core cpu) but memory usage is still a (huge) problem.
Any other suggestions?

[tango2k]

Oh, and I have a typeo, i have to open a 60 mb text file, not a 6mb (yeah, pretty useless to me, but I won't use the program !)

[webJose]

I have never done such a thing, so I can only suggest in plain English what my attempt would be:

I would keep in memory only the part of the file that is visible to the end user, that plus a bit before and after to keep things flowing.

One of the fundamental parts would be to trick the textbox's scrollbar into thinking there is more text in the box than what really is. Basically, you'll have to subclass the textbox. Handle the scrollbar notifications yourself, and keep dynamically loading from the file the parts that are needed. This should take care of the memory usage.

If you do this, then I wouldn't venture into multithreading to try to speed things up, because theoretically, you'll be only reading small chunks at a time. And even if I did multithreading, I would only do it knowing the computer is multi-cored. I would create one worker thread per core. No more. Maybe you can even set the affinity for each worker thread.

[helios]

I have a question. Will the user need to jump to any line in the file? Because if that's the case, you'll have no choice but to go through the entire file searching for newlines. Maybe it wouldn't be a bad idea to build an index file (in any other situation this would be silly, but we're dealing with a 60 MiB text file, here).

[tango2k]

Thank you webJose and helios for your answers!

Let me start with webJose:
I agree with you regarding multithreading (as already mentioned), so I could say I will probably discard doing such thing.
But I really like your trick. I think in a couple of days I will try to pul it! It seems like a valid choice.

helios:
I think it depends on the user. Some of them know what part of that huge file they are interested to look... but some of them don't. So it's tricky.

What really bothers me is the fact that when I use, for example, Wordpad, it opens as fast as a rocket!! Same thing with Notepad++ (and this one uses c++). Perhaps both of them use webJose's trick to get such speeds, right?

Right now, I am going to try webJose's suggestion.

Thanks everybody!! Will get back as soon as possible!

[george135]

Use Win32 MMF