CODEWITHSUNDEEP
pythonfiletesting
seanrose
seanrose

Reputation: 8705

Is there a Python module for creating test files of specific sizes?

I have a server that has files uploaded to it. I need to profile the upload/response time for various file sizes to that server i.e. how long it takes to upload a 10kb file, a 100mb file, and many other sizes. I want to avoid manually creating all of the files and storing them.

Is there a Python module that lets you create test files of arbitrary sizes? I'm basically looking for something that would work like:

test_1mb_file = test_file_module.create_test_file(size=1048576)

Upvotes: 4

Views: 9030

Answers (4)

Emilio
Emilio

Reputation: 81

From @abarnert and @jedwards answers:

mb = 1
with tempfile.TemporaryFile() as tf:
    tf.seek(mb * 1024 * 1024 - 1)
    tf.write(b'0')
    tf.seek(0)

Upvotes: 1

abarnert
abarnert

Reputation: 366133

You don't really need to write 1MB to create a 1MB file:

with open('bigfile', 'wb') as bigfile:
    bigfile.seek(1048575)
    bigfile.write('0')

On the other hand, do you really need a file at all? Many APIs take any "file-like object". It's not always clear whether that means read, read and seek, iterating by lines, or something else… but whatever it is, you should be able to simulate a 1MB file without creating more data than a single read or readline at a time.

PS, if you're not actually sending the files from Python, just creating them to use later, there are tools that are specifically designed for this kind of thing:

dd bs=1024 seek=1024 count=0 if=/dev/null of=bigfile # 1MB uninitialized
dd bs=1024 count=1024 if=/dev/zero of=bigfile # 1MB of zeroes
dd bs=1024 count=1024 if=/dev/random of=bigfile # 1MB of random data

Upvotes: 13

jedwards
jedwards

Reputation: 30250

I'd probably use something like

with tempfile.NamedTemporaryFile() as h:
    h.write("0" * 1048576)
    # Do whatever you need to do while the context manager keeps the file open
# Once you "outdent" the file will be closed and deleted.

This uses Python's tempfile module.

I used a NamedTemporaryFile in case you need external access to it, otherwise a tempfile.TemporaryFile would suffice.

Upvotes: 1

Joel Cornett
Joel Cornett

Reputation: 24788

Just do:

size = 1000
with open("myTestFile.txt", "wb") as f:
    f.write(" " * size)

Upvotes: 2

Related Questions