Reputation: 675
Self-modifying file: call versus import
Disclaimer: this is about weird language behavior in a crazy edge case. I'm using it to ask a bigger question about python and memory/disk read writes. How does dynamically editing a file and subsequently calling differ from importing the file and then calling? Is importing the same file different than exporting external modules with respect to loading from memory/ loading from disk? What conditions trigger reloading a function from disk.
I am trying to understand the way that python functions are loaded into memory (and when they are reread from disk). I wrote a simple script (temp.py) that modifies itself in a call to modify_this_function, it writes in a print line. Nothing crazy, predictable behavior, adds a print statement when called.
import dis
def modify_this_function():
f = open("temp.py", "r")
contents = f.readlines()
f.close()
contents.insert(3, '\tprint("hello!")\n')
f = open("temp.py", "w")
contents = "".join(contents)
f.write(contents)
f.close()
modify_this_function()
print(dis.dis(modify_this_function))
modify_this_function()
This call doesn't do anything particularly interesting (though it does modify the file on reload, adds two print statements). The output of the call to dis.dis reflects the original function definition.
4 0 LOAD_GLOBAL 0 (open)
3 LOAD_CONST 1 ('temp.py')
6 LOAD_CONST 2 ('r')
9 CALL_FUNCTION 2
12 STORE_FAST 0 (f)
5 15 LOAD_FAST 0 (f)
18 LOAD_ATTR 1 (readlines)
21 CALL_FUNCTION 0
24 STORE_FAST 1 (contents)
6 27 LOAD_FAST 0 (f)
30 LOAD_ATTR 2 (close)
33 CALL_FUNCTION 0
36 POP_TOP
8 37 LOAD_FAST 1 (contents)
40 LOAD_ATTR 3 (insert)
43 LOAD_CONST 3 (3)
46 LOAD_CONST 4 ('\tprint("hello!")\n')
49 CALL_FUNCTION 2
52 POP_TOP
10 53 LOAD_GLOBAL 0 (open)
56 LOAD_CONST 1 ('temp.py')
59 LOAD_CONST 5 ('w')
62 CALL_FUNCTION 2
65 STORE_FAST 0 (f)
11 68 LOAD_CONST 6 ('')
71 LOAD_ATTR 4 (join)
74 LOAD_FAST 1 (contents)
77 CALL_FUNCTION 1
80 STORE_FAST 1 (contents)
12 83 LOAD_FAST 0 (f)
86 LOAD_ATTR 5 (write)
89 LOAD_FAST 1 (contents)
92 CALL_FUNCTION 1
95 POP_TOP
13 96 LOAD_FAST 0 (f)
99 LOAD_ATTR 2 (close)
102 CALL_FUNCTION 0
105 POP_TOP
106 LOAD_CONST 0 (None)
109 RETURN_VALUE
None
This seems to indicate that the function was in memory, not reloaded from disk. Is that correct?
import dis
def modify_this_function():
f = open("temp.py", "r")
contents = f.readlines()
f.close()
contents.insert(3, '\tprint("hello!")\n')
f = open("temp.py", "w")
contents = "".join(contents)
f.write(contents)
f.close()
modify_this_function()
print(dis.dis(modify_this_function))
#modify_this_function()
from temp import modify_this_function
This function call is more interesting.
4 0 LOAD_GLOBAL 0 (open)
3 LOAD_CONST 1 ('temp.py')
6 LOAD_CONST 2 ('r')
9 CALL_FUNCTION 2
12 STORE_FAST 0 (f)
5 15 LOAD_FAST 0 (f)
18 LOAD_ATTR 1 (readlines)
21 CALL_FUNCTION 0
24 STORE_FAST 1 (contents)
6 27 LOAD_FAST 0 (f)
30 LOAD_ATTR 2 (close)
33 CALL_FUNCTION 0
36 POP_TOP
8 37 LOAD_FAST 1 (contents)
40 LOAD_ATTR 3 (insert)
43 LOAD_CONST 3 (3)
46 LOAD_CONST 4 ('\tprint("hello!")\n')
49 CALL_FUNCTION 2
52 POP_TOP
10 53 LOAD_GLOBAL 0 (open)
56 LOAD_CONST 1 ('temp.py')
59 LOAD_CONST 5 ('w')
62 CALL_FUNCTION 2
65 STORE_FAST 0 (f)
11 68 LOAD_CONST 6 ('')
71 LOAD_ATTR 4 (join)
74 LOAD_FAST 1 (contents)
77 CALL_FUNCTION 1
80 STORE_FAST 1 (contents)
12 83 LOAD_FAST 0 (f)
86 LOAD_ATTR 5 (write)
89 LOAD_FAST 1 (contents)
92 CALL_FUNCTION 1
95 POP_TOP
13 96 LOAD_FAST 0 (f)
99 LOAD_ATTR 2 (close)
102 CALL_FUNCTION 0
105 POP_TOP
106 LOAD_CONST 0 (None)
109 RETURN_VALUE
None
hello!
4 0 LOAD_CONST 1 ('hello!')
3 PRINT_ITEM
4 PRINT_NEWLINE
5 5 LOAD_GLOBAL 0 (open)
8 LOAD_CONST 2 ('temp.py')
11 LOAD_CONST 3 ('r')
14 CALL_FUNCTION 2
17 STORE_FAST 0 (f)
6 20 LOAD_FAST 0 (f)
23 LOAD_ATTR 1 (readlines)
26 CALL_FUNCTION 0
29 STORE_FAST 1 (contents)
7 32 LOAD_FAST 0 (f)
35 LOAD_ATTR 2 (close)
38 CALL_FUNCTION 0
41 POP_TOP
9 42 LOAD_FAST 1 (contents)
45 LOAD_ATTR 3 (insert)
48 LOAD_CONST 4 (3)
51 LOAD_CONST 5 ('\tprint("hello!")\n')
54 CALL_FUNCTION 2
57 POP_TOP
11 58 LOAD_GLOBAL 0 (open)
61 LOAD_CONST 2 ('temp.py')
64 LOAD_CONST 6 ('w')
67 CALL_FUNCTION 2
70 STORE_FAST 0 (f)
12 73 LOAD_CONST 7 ('')
76 LOAD_ATTR 4 (join)
79 LOAD_FAST 1 (contents)
82 CALL_FUNCTION 1
85 STORE_FAST 1 (contents)
13 88 LOAD_FAST 0 (f)
91 LOAD_ATTR 5 (write)
94 LOAD_FAST 1 (contents)
97 CALL_FUNCTION 1
100 POP_TOP
14 101 LOAD_FAST 0 (f)
104 LOAD_ATTR 2 (close)
107 CALL_FUNCTION 0
110 POP_TOP
111 LOAD_CONST 0 (None)
114 RETURN_VALUE
None
Here, it seems like the print is called and it appears in the disassembler output. So does this indicate that the import triggered a reread from disk? A reread that wasn't initiated by simply calling the function. Is my intuition here correct?
Upvotes: 0
Views: 88
Answers (1)
Reputation: 104752
You are loading your module twice. That's not normally possible, since Python caches module objects, but it can happen. This time it occurs because the first time you're loading it as the main module, named __main__. The second time you load it as its normal name temp. The second time, you'll see the results of the modifications the module made to its own file prior to the import that loaded again.
After a module is imported, changes to the file it was loaded from won't be reflected in the module's code, which was all compiled at the time the module was loaded. Such changes might confuse some debugging tools into reading the wrong version of the source and misreporting error locations and other details in some situations, but they won't have any effect on how the code runs.
You can also use the reload function (from importlib in modern versions of Python, a builtin in Python 2) to reload a module on demand. The contents of the new version of the module will overwrite the original version. Note though that any external references to objects from the old version of the module will remain pointing to the same old objects.
Upvotes: 1
Related Questions
- Python when are imported files evaluated?
- Handling imports in externally-called multi-file script
- Python - When to use file vs open
- Importing a py file within itself
- Should I open a file outside or inside a function?
- Python import vs direct execution
- Import vs open for input file
- Difference between opening a file and using a module
- What's the benefit of allowing python files to import themselves
- Is importing a file good in Python