C++ caching
Hi,
I need to store data of a 1MB file in cache, so when I close the application, and open it again, I can find the data.
Thank you.
I need to store data of a 1MB file in cache, so when I close the application, and open it again, I can find the data.
Thank you.
I need to store in cache. not in a temporary memory. I need to read file once to memory, and then read directly from memory.
Thanks.
Thanks.
the extra simple workaround to this type of requirement is ye olde ram disk, which are still available. It makes a block of ram look like a disk drive. If you reboot the machine, all items in the 'drive' are deleted, so be sure to save changes back out to a real disk at some sensible frequency. I think some of the modern drivers for these can optionally cache and restore the contents to disk on demand and read it in at startup.
you can also use a memory mapped file / memory based process communications techniques, but I am not an expert in those and don't know how to make them persist after program termination.
there are other methods if this does not work for you.
-- you cannot force something to stay in cache. CPU cache is a precious resource and I am not aware of any way to lock it down, and even if you could, it would probably cause problems on any operational system and could only be safely done on a dedicated / embedded machine possibly with a modified OS that let you do this.
on any modern machine a 1mb file is so small it can be read from (a defragmented or SSD) disk and put into memory incredibly fast. you may not get a lot of performance from this approach; every scenario I can think of where this would give appreciable performance gains imply a design problem.
you can also use a memory mapped file / memory based process communications techniques, but I am not an expert in those and don't know how to make them persist after program termination.
there are other methods if this does not work for you.
-- you cannot force something to stay in cache. CPU cache is a precious resource and I am not aware of any way to lock it down, and even if you could, it would probably cause problems on any operational system and could only be safely done on a dedicated / embedded machine possibly with a modified OS that let you do this.
on any modern machine a 1mb file is so small it can be read from (a defragmented or SSD) disk and put into memory incredibly fast. you may not get a lot of performance from this approach; every scenario I can think of where this would give appreciable performance gains imply a design problem.
Last edited on
Jonnin,
Thanks for your help. I tried to save a variable in memory, and get the address of that variable, and then tried to remove the initialization of the variable, and then print what inside the address, and it did change (Memory not persistent). Is there any way to make it persistent.
Example
int x=0; has address &aabbccdd
next time I run the program, I want to get the data inside aabbccdd without initialization of x;
I need to print data in aabbccdd, and it should give me 0.
Thanks.
Thanks for your help. I tried to save a variable in memory, and get the address of that variable, and then tried to remove the initialization of the variable, and then print what inside the address, and it did change (Memory not persistent). Is there any way to make it persistent.
Example
int x=0; has address &aabbccdd
next time I run the program, I want to get the data inside aabbccdd without initialization of x;
I need to print data in aabbccdd, and it should give me 0.
Thanks.
not using normal programming.
it works more or less (OS depends, details ignored) like this;
you start your program.
your program is given a big wad of memory from the OS. This is the stack.
your program may elect to put some memory outside this space, the heap.
your program does things to some memory locations.
your program exits.
the operating system cleans up after your program (most do, a very few do NOT, mostly antiques like dos 1.0). The memory you had changed is now marked as usable by the OS again.
the os gives 'your' memory to 'something else'
something else modifies that memory (or not, luck of the draw here**).
your new program starts.
your program attempts to access the memory location.
-- your smart OS thinks this is virus-like or bugged behavior and crashes your program outright or
-- your dumb os allows it but the value may have changed anyway due to something else (** above)
and so on.
you can't force the OS to not do its tasks with normal C++ and certainly you can't force this while the program that is trying to do the forcing isnt even running anymore.
the only way I know to do THAT is to customize your OS memory management, which is nontrivial and would imply compiling a special version of unix, or again, some weird embedded OS.
you can leave a background process running and do interprocess communication with it from your other program. All the background process would do is lay claim to the memory, expose it to your other program, and then go idle. This isnt what you asked for, as the memory owning program doesn't exit.
or you can use the ram disk, which functions as the idle program I mentioned, holding the memory open for use by your programs, and you interface to the values as if reading binary or text files but without any disk overhead.
If someone here knows a way to hold the memory after exiting a program ... hopefully they will share and we can both learn something...
it works more or less (OS depends, details ignored) like this;
you start your program.
your program is given a big wad of memory from the OS. This is the stack.
your program may elect to put some memory outside this space, the heap.
your program does things to some memory locations.
your program exits.
the operating system cleans up after your program (most do, a very few do NOT, mostly antiques like dos 1.0). The memory you had changed is now marked as usable by the OS again.
the os gives 'your' memory to 'something else'
something else modifies that memory (or not, luck of the draw here**).
your new program starts.
your program attempts to access the memory location.
-- your smart OS thinks this is virus-like or bugged behavior and crashes your program outright or
-- your dumb os allows it but the value may have changed anyway due to something else (** above)
and so on.
you can't force the OS to not do its tasks with normal C++ and certainly you can't force this while the program that is trying to do the forcing isnt even running anymore.
the only way I know to do THAT is to customize your OS memory management, which is nontrivial and would imply compiling a special version of unix, or again, some weird embedded OS.
you can leave a background process running and do interprocess communication with it from your other program. All the background process would do is lay claim to the memory, expose it to your other program, and then go idle. This isnt what you asked for, as the memory owning program doesn't exit.
or you can use the ram disk, which functions as the idle program I mentioned, holding the memory open for use by your programs, and you interface to the values as if reading binary or text files but without any disk overhead.
If someone here knows a way to hold the memory after exiting a program ... hopefully they will share and we can both learn something...
Last edited on
RAM Disks just make it so it loads the file when the OS boots as opposed to loading the file when accessed in the program, doesn't it? I haven't actually used it.
No one has asked why anasmourad wants to do this. So, anasmourad, can you tell us why?
In addition to what jonnin said, note that a program has its own virtual memory, which is then independently mapped to real memory. In other words, two programs can both use the address 3000, but that number shouldn't actually mean anything to the user. The address of a pointer is an implementation-level detail, its actual value (aside from not being NULL) should not be relied on!
If I were forced to do something like this, I would do the "background process" (or service/daemon) that jonnin mentioned. This background process could start when the computer started (or when you first launch your program). It is this background process that keeps the memory you want in memory, which persists between runs of the foreground program. Need to use IPC.
No one has asked why anasmourad wants to do this. So, anasmourad, can you tell us why?
In addition to what jonnin said, note that a program has its own virtual memory, which is then independently mapped to real memory. In other words, two programs can both use the address 3000, but that number shouldn't actually mean anything to the user. The address of a pointer is an implementation-level detail, its actual value (aside from not being NULL) should not be relied on!
If I were forced to do something like this, I would do the "background process" (or service/daemon) that jonnin mentioned. This background process could start when the computer started (or when you first launch your program). It is this background process that keeps the memory you want in memory, which persists between runs of the foreground program. Need to use IPC.
Last edited on
What are you actually trying to achieve? What data do you need to persist between calls to your program, and why can you not use a file to store it? That is the standard solution.
As Jonin says, when your program ends, all the memory it had allocated to it is taken back by the OS. In windows, anything in that memory gets wiped.
As Jonin says, when your program ends, all the memory it had allocated to it is taken back by the OS. In windows, anything in that memory gets wiped.
Last edited on
Ganado, a ram disk is a driver program that turns a block of ram into a filesystem so you can put a few files in there for very fast access. It looks exactly like another hard disk to your OS/system. The drivers vary in capability a bit; some load from a disk file to memory and back again on startup/shutdown, and others just make the space and you load them manually. They were popular for caching floppy disks to ram due to the extra special incredible slowness of reading floppies back when, and came back into a little bit of popularity in modern times now that we have 'an excess' of ram (relatively speaking, but no one is going to miss a 100MB chunk anymore for example). I had an 'unsafe' web browsing sandbox that put all the temp internet files in one, and it was an amazing experience -- all the junk cleared for sure on a reboot, and fast access to it when running. I may need to do that again :)
its basically identical to IPC except using an ifstream / ofstream interface to get at the data. Since he said a 1mb *file* initially, its already in a *file* format so this made sense to me earlier. The second post about addressing left me mystified.
its basically identical to IPC except using an ifstream / ofstream interface to get at the data. Since he said a 1mb *file* initially, its already in a *file* format so this made sense to me earlier. The second post about addressing left me mystified.
Last edited on
Topic archived. No new replies allowed.