CODEWITHSUNDEEP
python
hab
hab

Reputation: 1

writing a string from one very long file to another file in python

Please do not behead me for my noob question. I have looked up many other questions on stackoverflow concerning this topic, but haven't found a solution that works as intended.

The Problem: I have a fairly large txt-file (about 5 MB) that I want to copy via readlines() or any other build in string-handling function into a new file. For smaller files the following code sure works (only schematically coded here):

f = open('C:/.../old.txt', 'r');
n = open('C:/.../new.txt', 'w');
for line in f:
    print(line, file=n);

However, as I found out here (UnicodeDecodeError: 'charmap' codec can't encode character X at position Y: character maps to undefined), internal restrictions of Windows prohibit this from working on larger files. So far, the only solution I came up with is the following:

f = open('C:/.../old.txt', 'r', encoding='utf8', errors='ignore');
n = open('C:/.../new.txt', 'a');
for line in f:
    print(line, file=sys.stderr) and append(line, file='C:/.../new.txt');   

f.close();
n.close();

But this doesn't work. I do get a new.txt-file, but it is empty. So, how do I iterate through a long txt-file and write every line into a new txt-file? Is there a way to read the sys.stderr as the source for the new file (I actually don't have any idea, what this sys.stderr is)? I know this is a noob question, but I don't know where to look for an answer anymore.

Thanks in advance!

Upvotes: 0

Views: 57

Answers (4)

pravin lanjile
pravin lanjile

Reputation: 1

you can try with this code it works for me.

with open("file_path/../large_file.txt") as f:
    with open("file_path/../new_file", "wb") as new_f:
            new_f.writelines(f.readlines())
            new_f.close()
    f.close()

Upvotes: 0

sureshvv
sureshvv

Reputation: 4422

Why don't you just use the shutil module and copy the file?

Upvotes: 0

AChampion
AChampion

Reputation: 30258

There is no need to use print() just write() to the file:

with open('C:/.../old.txt', 'r') as f, open('C:/.../new.txt', 'w') as n:
    n.writelines(f)

However, it sounds like you may have an encoding issue, so make sure that both files are opened with the correct encoding. If you provide the error output perhaps more help can be provided.

BTW: Python doesn't use ; as a line terminator, it can be used to separate 2 statements if you want to put them on the same line but this is generally considered bad form.

Upvotes: 1

Dongeon Kim
Dongeon Kim

Reputation: 473

You can set standard output to file like my code. I successfully copied 6MB text file with this.

import sys

bigoutput = open("bigcopy.txt", "w")
sys.stdout = bigoutput
with open("big.txt", "r") as biginput:
    for bigline in biginput.readlines():
        print(bigline.replace("\n", ""))
bigoutput.close()

Upvotes: 0

Related Questions