CODEWITHSUNDEEP
bash
user121196
user121196

Reputation: 31070

change a few bytes in a large file without loading everything in memory using bash on linux

in bash on linux is there a way to change a few bytes/characters in a large file? for example, seek to position x and then optionally delete a few characters, and then insert a few characters. I prefer to do this in bash.

Upvotes: 1

Views: 1105

Answers (1)

Stephen C
Stephen C

Reputation: 719679

In general, it is not possible to make an "edit" to a file that changes the position of characters after the edit point ... except by reading and rewriting all characters after the edit point.

The reason for this is fundamental to the way that files are represented, and the way that file system APIs work. And these in turn derive from the way that the physical storage devices work.

So a general solution would need to be implemented something like this (pseudo-code):

# Replace N bytes starting at position P with bytes B1, B2,...

open file in "random access, no truncation" mode 
seek to N + P
read remainder of file into a buffer.
seek to N
write bytes B1, B2, ...
write bytes from the buffer
close

(You could avoiding the entire "remainder of the file" into a buffer, but the logic is more complicated. And tangential to what I'm trying to explain ...)

Anyway, I'm not aware of an existing utility that will do the above, but you could write an ad-hoc program to do that if you wanted to.

If the number of bytes you were replacing was exactly the same as the number of replacement bytes, you could do this with an in-place update. According to How to overwrite some bytes of a binary file with dd? you can do this with the "dd" command. (The Answer demonstrates a 1 byte in-place update.)

Upvotes: 2

Related Questions