Reading and Writing to the same file in c++

Question

I am hoping this is an easy question that someone can answer for me or give me an example. I am working with files that can be sized upwards of 4GB and I foresee memory issues if I want to store my entire input file edit it and then output it, so I thought it would be easier to look into rewriting the file as I go (line by line or more likely section of 25 lines at a time). As I looked into it though it seems way more complicated than I originally thought, and everywhere (including this site) people recommend storing the data or opening a separate output file. Is it possible to edit a file as you read it in? If so what is the best way to do so? Should I try to use the npos of the file to go back to what a read in?

File Format (Without header and extraneous information):

* voxel 0 0 0 1 1 1 3
Res 000000000000 000100000000 2.66668e+06
Cap 000000000000 000100000000 2.19141e-16
Res 000000010000 000100010000 2.66668e+06
Cap 000000010000 000100010000 2.19141e-16
Res 000000000001 000100000001 2.66668e+06
Cap 000000000001 000100000001 2.19141e-16
Res 000000010001 000100010001 2.66668e+06
Cap 000000010001 000100010001 2.19141e-16
Res 000000000000 000000010000 2.66668e+06
Cap 000000000000 000000010000 2.19141e-16
Res 000100000000 000100010000 2.66668e+06
Cap 000100000000 000100010000 2.19141e-16
Res 000000000001 000000010001 2.66668e+06
Cap 000000000001 000000010001 2.19141e-16
Res 000100000001 000100010001 2.66668e+06
Cap 000100000001 000100010001 2.19141e-16
Res 000000000000 000000000001 2.66668e+06
Cap 000000000000 000000000001 2.19141e-16
Res 000100000000 000100000001 2.66668e+06
Cap 000100000000 000100000001 2.19141e-16
Res 000000010000 000000010001 2.66668e+06
Cap 000000010000 000000010001 2.19141e-16
Res 000100010000 000100010001 2.66668e+06
Cap 000100010000 000100010001 2.19141e-16

Based on information from another file and the location (the 0 0 0 1 1 1) at the top of the file, determines how and what values change, but again I am really concerned when talking about thousands if not more of these blocks of data that I cannot read from and then write to a new file, and I have no idea how to read and write effectively on this file. The only thing that should change is the values at the end of each line (2.6668e+06 and 2.19141e-16) although those will differ, they are all the same in this case for easier understanding. I am currently reading in the file (ifstream only) and can get to the point where I need to rewrite the file but don't know how to easily change position of where I am writing nor how to insert over vs just insert in and other similar issues of writing to an existing file.

Any advice is appreciated, short examples especially so!

Answer 1

To expand on what Mats Petersson said, with files that size, and with like-sized writes, you would be well served by memory-mapped files--especially with the file sizes you're describing. Otherwise, your next best bet is going to be buffer-list scheme, which doesn't necessarily have to be much more complicated.

Answer 2

If the records you are modifying are always the same size for both "new" and "old" data, it's no problem rewriting into the same file - it's only an issue if you are going to write data that is a different length than your "old" data.

Just open the file with fstream f("somename.ext", ios::out|ios::in), and use f.seekg() and f.seekp() as required to go to the relevant place in the file (you can use tellp and tellg to figure out where you are currently)