Code1:
loop begin
Read data point from file
Process and store in array
loop end
Output array to file on disk
Code2:
loop begin
Read data point from file
Process and output point to file on disk
loop end
Code3:
Read data points from file into array
loop begin
Process and store in array
loop end
Output array to file on disk
Will there be any performance difference between the above approaches? Would it relate to number of data? I read there is a buffer that stores the output before it is synchronized to disk; does the compiler do any 'optimization' when the file IO is within a loop? Would this become an issue in embedded systems programming where memory might be at a premium?
Always try to cluster I/O operations. If the entire file fits in memory, read the entire file to memory. Otherwise, read chunks of it at a time. I would suggest no less than 4K.