Reputation: 1323
When writing to a file in python, you should typically use the using structure in order to have the file closed after writing, like so:
with open("myfile.txt", "a") as file1:
file1.write("Hello \n")
But if I, during the execution of my script, wants to write to the same file from different places in the code, I might be tempted to encapsulate the above structure in some kind of method like so:
def write_to_file(self, string_to_write):
with open(self.myfile_path, "w") as file1:
file1.write(f"{string_to_write}\n")
The above would likely give me a pretty bad performance hit since the file is opened every time I call the method.
The alternative that I can see is opening the file early in the program and having a file.close() call in some finally clause somewhere and hope for the best. But I understand this to be associated with some risk.
So given the above, how should one approach this task in a pythonic as well as a performant way?
Upvotes: 0
Views: 237
Reputation: 27660
Unless you can put everything into with
block like in scotscotmcc's answer, I'd probably just do it without with
. Just open
the file, write to it, then .close()
it.
But here's an idea for using with
and not putting the code into the with
block. A generator that writes to the file what you send it:
def printer(filename):
with open(filename, 'w') as f:
while True:
print((yield), file=f)
# Demo usage
p = printer('test.txt')
next(p)
p.send('foo')
p.send('bar')
p.close()
# Check the resulting file
with open('test.txt') as f:
print(repr(f.read()))
Output (Try it online!):
'foo\nbar\n'
I don't think it's any better than just open
+close
a file, though. I just tried it for fun / out of curiosity.
Upvotes: 0
Reputation: 1323
Based on @CharlieBONS comment and some Singleton patterns for python, I created this solution. I still suspect there is something better, closer to the file loggers in logging, but I have not had the time to dive into that.
The previous solution was based on logging to a file with a specific logging level, but since the python library needs to hook into other solutions with their own logging configurations, like AirFlow, i had to abandon this.
import json
import logging
import os
from pathlib import Path
from typing import List
class MyWriter:
__instance = None
__inited = False
def __new__(cls, path_to_file: Path) -> "MyWriter":
if cls.__instance is None:
cls.__instance = super().__new__(cls)
return cls.__instance
def __init__(self, path_to_file: Path) -> None:
if type(self).__inited:
return
self.cache: List[str] = []
self.path_to_file: Path = path_to_file
if self.path_to_file.is_file():
os.remove(self.path_to_file)
type(self).__inited = True
def write(self, string_to_write: str, flush=False):
try:
if string_to_write:
self.cache.append(f"{string_to_write)}\n")
if len(self.cache) > 1000 or flush:
with open(self.path_to_file, "a") as my_file:
extradata_file.writelines(self.cache)
self.cache = []
logging.debug("My Writer flushing the cache")
except Exception as ee:
error_message = "Something went wrong in My Writer"
logging.error(error_message)
raise ee
def flush(self):
self.write("", True)
Upvotes: 0
Reputation: 3133
You can have your with
statement early in your code and have all the other functions that use the file (and probably many that don't use the file but are called in between the ones that do) indented from it and pass the file to them.
This may not be wonderful to refactor things to this given the current code and complexity...
def main():
with open('file.txt','w') as file:
my_func_1()
my_func_2(file)
my_func_3
my_func_4(file)
...
def my_func_1():
...
def my_func_2(file):
...
file.write('thing to write')
...
def my_func_3():
...
def my_func_4(file):
...
file.write('thing to write')
...
Upvotes: 2