CODEWITHSUNDEEP
pythonpython-2.7
Ziva
Ziva

Reputation: 3501

Python: saving in file big entries

I want to save in a file (don't mind the type) some string informations about entries and objects. I can generate the string from my data and I know how to save a string in a .txt file. However, when I'm reading from the file, I'm reading "lines", so I'm assuming it reads a line till the first new line symbol. However, some of my strings are longer than a document line and when I want to read it, I get errors. How I can save a long string in a file to not loose any data?

It is, how I'm saving in the file:

with codecs.open(filename, 'a', "utf-8") as outfile:
    outfile.write(data_string + '\n')

and how I read the data from the file:

with codecs.open(filename, 'r',"utf-8") as infile:
    lines = infile.readlines()

Upvotes: 0

Views: 40

Answers (1)

Wayne Werner
Wayne Werner

Reputation: 51837

You have a few options:

dump/load as JSON

import tempfile
import json

text = 'this is my string with a \nnewline in it'

with tempfile.TemporaryFile(mode='w+') as f:
    f.write(json.dumps([text]))
    f.flush()
    f.seek(0)
    lines = json.load(f)
    print(lines)

Downsides: JSON may be fairly human readable, but a little error in your file will bork everything up. Not nearly as legible as plain ol' text.

pickle

import tempfile
import pickle

text = 'this is my string with a \nnewline in it'

with tempfile.TemporaryFile(mode='w+') as f:
    f.write(pickle.dumps([text]))
    f.flush()
    f.seek(0)
    lines = pickle.load(f)
    print(lines)

Downsides: pickle is terribly insecure and you should pretty much treat it as you would eval. If you wouldn't feel comfortable using eval in that situation, then you shouldn't use pickle.

Your own sentinel

import tempfile

text = 'this is my string with a \nnewline in it'
other_text = 'this line has no newline in it'

with tempfile.TemporaryFile(mode='w+') as f:
    f.write(text)
    f.write(chr(30))
    f.write(other_text)
    f.flush()
    f.seek(0)
    lines = f.read().split(chr(30))
    print(lines)

Downsides: Could be a little trickier. You have to make sure that your sentinel isn't found in the text itself. Also you can't use readlines, and iterating line by line is made a bit more awkward.

Upvotes: 1

Related Questions