CODEWITHSUNDEEP
pythonnumpy
Kristofer
Kristofer

Reputation: 1487

Append element to binary file

My goal is: Open a binary file, store (append) every 100 values into this file, and continue doing so until finishing. To do so, I use the following simple loop to simulate that:

import numpy as np
import random

alist=[]
c = 1

for i in range(1000):
    alist.append(i)
    if i == (c*100):
        np.array(alist).tofile("file.bin")
        print alist
        c = c + 1
        alist[:] = [] # clear the list before continuing

However, when I check the size of file.bin, I feel numpy does not append rather it is replacing which is not what I want. How to fix that?

Thank you.

Upvotes: 3

Views: 3823

Answers (1)

AGN Gazer
AGN Gazer

Reputation: 8378

Of course numpy is replacing/overwriting your old data every time you (re-)open the file for writing. This is almost universal behavior of most tofile() like functions (and not only in numpy).

Solution: Open a file handle for writing before the loop and pass that to tofile() function. Like this:

import numpy as np
import random

alist=[]
c = 1
with open("file.bin", "wb") as f: # or choose 'w+' mode - read "open()" documentation
    for i in range(1000):
        alist.append(i)
        if i == (c*100):
            np.array(alist).tofile(f)
            print alist
            c = c + 1
            alist[:] = [] # clear the list before continuing

Now the code opens the file before entering the loop and tofile() method re-uses an already opened file handle instead of re-opening and thus overwriting an existing file (created in the run of the loop).

Upvotes: 4

Related Questions