Insert line at middle of file with Python?

Question

Is there a way to do this? Say I have a file that's a list of names that goes like this:

1. Alfred
2. Bill
3. Donald

How could I insert the third name, "Charlie", at line x (in this case 3), and automatically send all others down one line? I've seen other questions like this, but they didn't get helpful answers. Can it be done, preferably with either a method or a loop?

Answer 1

If you want to search a file for a substring and add a new text to the next line, one of the elegant ways to do it is the following:

import os, fileinput

old = "A"
new = "B"

for line in fileinput.FileInput(file_path, inplace=True):
    if old in line :
        line += new + os.linesep
    print(line, end="")

Answer 2

I write this in order to reutilize/correct martincho's answer (accepted one)

! IMPORTANT: This code loads all the file into ram and rewrites content to the file

Variables index, value may be what you desire, but pay attention to making value string and end with '\n' if you don't want it to mess with existing data.

with open("path_to_file", "r+") as f:
    # Read the content into a variable
    contents = f.readlines()
    contents.insert(index, value)
    
    # Reset the reader's location (in bytes)
    f.seek(0)
    
    # Rewrite the content to the file
    f.writelines(contents)

See the python docs about file.seek method: Python docs

Answer 3

This is a way of doing the trick.

with open("path_to_file", "r") as f:
    contents = f.readlines()

contents.insert(index, value)

with open("path_to_file", "w") as f:
    contents = "".join(contents)
    f.write(contents)

index and value are the line and value of your choice, lines starting from 0.

Answer 4

The accepted answer has to load the whole file into memory, which doesn't work nicely for large files. The following solution writes the file contents with the new data inserted into the right line to a temporary file in the same directory (so on the same file system), only reading small chunks from the source file at a time. It then overwrites the source file with the contents of the temporary file in an efficient way (Python 3.8+).

from pathlib import Path
from shutil import copyfile
from tempfile import NamedTemporaryFile

sourcefile = Path("/path/to/source").resolve()
insert_lineno = 152  # The line to insert the new data into.
insert_data = "..."  # Some string to insert.

with sourcefile.open(mode="r") as source:
    destination = NamedTemporaryFile(mode="w", dir=str(sourcefile.parent))
    lineno = 1

    while lineno < insert_lineno:
        destination.file.write(source.readline())
        lineno += 1

    # Insert the new data.
    destination.file.write(insert_data)

    # Write the rest in chunks.
    while True:
        data = source.read(1024)
        if not data:
            break
        destination.file.write(data)

# Finish writing data.
destination.flush()
# Overwrite the original file's contents with that of the temporary file.
# This uses a memory-optimised copy operation starting from Python 3.8.
copyfile(destination.name, str(sourcefile))
# Delete the temporary file.
destination.close()

EDIT 2020-09-08: I just found an answer on Code Review that does something similar to above with more explanation - it might be useful to some.

Answer 5

location_of_line = 0
with open(filename, 'r') as file_you_want_to_read:
     #readlines in file and put in a list
     contents = file_you_want_to_read.readlines()

     #find location of what line you want to insert after
     for index, line in enumerate(contents):
            if line.startswith('whatever you are looking for')
                   location_of_line = index

#now you have a list of every line in that file
context.insert(location_of_line, "whatever you want to append to middle of file")
with open(filename, 'w') as file_to_write_to:
        file_to_write_to.writelines(contents)

That is how I ended up getting whatever data I want to insert to the middle of the file.

this is just pseudo code, as I was having a hard time finding clear understanding of what is going on.

essentially you read in the file to its entirety and add it into a list, then you insert your lines that you want to that list, and then re-write to the same file.

i am sure there are better ways to do this, may not be efficient, but it makes more sense to me at least, I hope it makes sense to someone else.

Answer 6

Below is a slightly awkward solution for the special case in which you are creating the original file yourself and happen to know the insertion location (e.g. you know ahead of time that you will need to insert a line with an additional name before the third line, but won't know the name until after you've fetched and written the rest of the names). Reading, storing and then re-writing the entire contents of the file as described in other answers is, I think, more elegant than this option, but may be undesirable for large files.

You can leave a buffer of invisible null characters ('\0') at the insertion location to be overwritten later:

num_names = 1_000_000    # Enough data to make storing in a list unideal
max_len = 20             # The maximum allowed length of the inserted line
line_to_insert = 2       # The third line is at index 2 (0-based indexing)

with open(filename, 'w+') as file:
    for i in range(line_to_insert):
        name = get_name(i)                    # Returns 'Alfred' for i = 0, etc.
        file.write(F'{i + 1}. {name}\n')

    insert_position = file.tell()             # Position to jump back to for insertion
    file.write('\0' * max_len + '\n')         # Buffer will show up as a blank line

    for i in range(line_to_insert, num_names):
        name = get_name(i)
        file.write(F'{i + 2}. {name}\n')      # Line numbering now bumped up by 1.

# Later, once you have the name to insert...
with open(filename, 'r+') as file:            # Must use 'r+' to write to middle of file 
    file.seek(insert_position)                # Move stream to the insertion line
    name = get_bonus_name()                   # This lucky winner jumps up to 3rd place
    new_line = F'{line_to_insert + 1}. {name}'
    file.write(new_line[:max_len])            # Slice so you don't overwrite next line

Unfortunately there is no way to delete-without-replacement any excess null characters that did not get overwritten (or in general any characters anywhere in the middle of a file), unless you then re-write everything that follows. But the null characters will not affect how your file looks to a human (they have zero width).

Answer 7

A simple but not efficient way is to read the whole content, change it and then rewrite it:

line_index = 3
lines = None
with open('file.txt', 'r') as file_handler:
    lines = file_handler.readlines()

lines.insert(line_index, 'Charlie')

with open('file.txt', 'w') as file_handler:
    file_handler.writelines(lines)

Answer 8

There is a combination of techniques which I found useful in solving this issue:

with open(file, 'r+') as fd:
    contents = fd.readlines()
    contents.insert(index, new_string)  # new_string should end in a newline
    fd.seek(0)  # readlines consumes the iterator, so we need to start over
    fd.writelines(contents)  # No need to truncate as we are increasing filesize

In our particular application, we wanted to add it after a certain string:

with open(file, 'r+') as fd:
    contents = fd.readlines()
    if match_string in contents[-1]:  # Handle last line to prevent IndexError
        contents.append(insert_string)
    else:
        for index, line in enumerate(contents):
            if match_string in line and insert_string not in contents[index + 1]:
                contents.insert(index + 1, insert_string)
                break
    fd.seek(0)
    fd.writelines(contents)

If you want it to insert the string after every instance of the match, instead of just the first, remove the else: (and properly unindent) and the break.

Note also that the and insert_string not in contents[index + 1]: prevents it from adding more than one copy after the match_string, so it's safe to run repeatedly.

Answer 9

You can just read the data into a list and insert the new record where you want.

names = []
with open('names.txt', 'r+') as fd:
    for line in fd:
        names.append(line.split(' ')[-1].strip())

    names.insert(2, "Charlie") # element 2 will be 3. in your list
    fd.seek(0)
    fd.truncate()

    for i in xrange(len(names)):
        fd.write("%d. %s\n" %(i + 1, names[i]))

Answer 10

1. Parse the file into a python list using file.readlines() or file.read().split('\n')
2. Identify the position where you have to insert a new line, according to your criteria.
3. Insert a new list element there using list.insert().
4. Write the result to the file.

Answer 11

You don't show us what the output should look like, so one possible interpretation is that you want this as the output:

1. Alfred
2. Bill
3. Charlie
4. Donald

(Insert Charlie, then add 1 to all subsequent lines.) Here's one possible solution:

def insert_line(input_stream, pos, new_name, output_stream):
  inserted = False
  for line in input_stream:
    number, name = parse_line(line)
    if number == pos:
      print >> output_stream, format_line(number, new_name)
      inserted = True
    print >> output_stream, format_line(number if not inserted else (number + 1), name)

def parse_line(line):
  number_str, name = line.strip().split()
  return (get_number(number_str), name)

def get_number(number_str):
  return int(number_str.split('.')[0])

def format_line(number, name):
  return add_dot(number) + ' ' + name

def add_dot(number):
  return str(number) + '.'

input_stream = open('input.txt', 'r')
output_stream = open('output.txt', 'w')

insert_line(input_stream, 3, 'Charlie', output_stream)

input_stream.close()
output_stream.close()