Open fstream with write, without clearing file

[LB]

I have a serious problem here: I need to open a file and overwrite existing data in the file. I cannot load the file into memory (the files being dealt with here are huge) so I was hoping to just deal with the file stream. Opening with ios::out clears the file as if it includes ios::trunc, and adding ios::app will only allow me to append to the end of the file, even if I move the write pointer somewhere else. Does anyone know what I can do?

[darkhorse]

Wouldn't you want to clear the file when you open it, or are you opening the file to overwrite data, and also need to do some operations with it?

[LB]

I want to modify the existing data in the file in the same file stream.

[ne555]

I cannot load the file into memory (the files being dealt with here are huge)

Then don't load all the file.
You can std::ofstream out( name, std::ios::out | std::ios::in ); to avoid the trunc, however remember that the size of the file will not change (you can modify, but not add/delete randomly)

[LB]

And then how do I reinsert the modified data back into the file at the same point I got it from?

[ne555]

while( in.read_block(block) ){
  process_block(block);
  out.write_block(block);
}
rename( out.name, in.name );

Or don't work with plain text

[LB]

I only need to process one part of the file and not deal with the rest...I thought this would be simple because writing a file from scratch is simple, can't I just overwrite part of what is written already like when first writing a file??

[Janlan]

Try ios::app

[LB]

Then even if I move the pointer elsewhere, it only appends to the file. There is no way to write anywhere else than the end of the file with ios::app it seems...

[shacktar]

This may sound like a hack, but...

1. Let's say you have a file, F, that has a size S in bytes.
2. Now, say you want to make a modification that requires you to overwrite F, starting at byte B, with your modification being M bytes long.

You will have to make a new file, F' (unfortunately). First, dump into F' bytes 0 to B-1 from F. Then, place your modification (M bytes) into F'. Now, you have two options. You can write the bytes from B+(M+1) to byte S of F to F' (this would be as if you had "insert" on). Otherwise, you can write the bytes from B to byte S of F to F' (this would be as if you had "insert" off). Now, delete F. F' will contain the modified file. Of course you may not be able to write all bytes at once; you could pick a batch size and write batches of bytes.

Of course this is an expensive operation for large files. You could do multiple modifications at once with an extension to the above. If your files are too big to handle for file streams (their file pointers reach a limit eventually), you would have to use OS-specific file functions.

[LB]

The files are large in that copying data takes more than a few minutes, but small enough that they can be used with file streams. It just doesn't make sense that I can't overwrite an existing part of a file...

[shacktar]

Well, you can overwrite an existing part of a file. It's just that the bytes won't shift over automatically for you (I think this is the key point).

You need to either work with temporary files (like my example above), or seek to the part of the file where you want to make the modification, copy the rest of the file, put your modification and append the rest.

[LB]

I want to overwrite so not being able to insert is fine. Copying the entire file over like that is taking several minutes; if I could just modify a part of the file and close it it'd be great. I'm just going to have to look outside the standard library for this, I guess.

[Null]

Maybe I'm missing something but isn't this what you want?

int main()
{

fstream file("file.txt",ios::in|ios::out);
string data;

// Read data
file.seekg(4);
file>>data;

// modify
data="one";

// write to file
file.seekp(4);
file<<data;

file.close();


}

unmodifed file.txt

one two three

modifed file.txt

one one three

[LB]

Except for the part where opening with ios::out empties the file completely, you have exactly what I want :p I can't understand the usage of ios::trunc if the file is emptied anyway with ios::out :\
Also I think you have to use ios::binary for seeking and telling the pointer.

I'm not sure how, but I recompiled the project in question and it appears that it is not clearing the file after all...it must've been something else. Sorry for all the trouble!

[Null]

Except for the part where opening with ios::out empties the file completely, you have exactly what I want :p

ne555 wrote:

You can std::ofstream out( name, std::ios::out | std::ios::in ); to avoid the trunc

Also I think you have to use ios::binary for seeking and telling the pointer.

Yeah, I think you are right.

[LB]

It was my project - somehow recompiling everything fixed the problem. I always have the most bizarre problems if I don't do a clean recompile of everything >_>