CODEWITHSUNDEEP
clinuxfile
hes_theman
hes_theman

Reputation: 628

How to remove a header from file without creating additional temporary files?

I'm developing an application for embedded system running Linux.

In my case, I have a quite big file (compared to the capability of the system) as input. The file has a small header whose size is just a few hundred bytes. In my application, I need to remove that header from the file so the file will have no header and contain only relevant data. Normally, I'd implement like following (pseudo code):

char *input_file  = "big_input.bin";
char *tmp_file1 = "header.bin";
char *tmp_file2 = "data.bin";
/* Copy the content of header from input file to tmp_file1 */ 
_copy_header(tmp_file1, input_file); 
/* Copy the data from input file to tmp_file2 */ 
_copy_data(tmp_file2, input_file);
/* Rename temp file to input file */
unlink(input_file);
rename(tmp_file2, input_file);

The problem with this approach is that it creates a temporary file tmp_file2 whose size is almost as big as input file (because the header is very small). In my system, everything is stored on RAM, which is very limited. Creating a big temporary file causes an out-of-memory error.

So how can I avoid creating a big temporary file?

Upvotes: 3

Views: 550

Answers (3)

alk
alk

Reputation: 70971

Open the same file twice, once for reading, once for writing.

Seek the read pointer behind the header.

Read from the read-pointer and write to the write-pointer.

Make sure the size you read and write at once is not larger than the header.

Cut off the header's size at the end of the file.

Upvotes: 6

Andrew Henle
Andrew Henle

Reputation: 1

Assuming you know the exact size of the header in advance, something like this should do it:

#define HEADER_SIZE 128

// size the buffer as appropriate for you RAM limits
char buffer[ 4096 ];
int fd = open( filename, O_RDWR );
size_t totalBytes = 0UL;
for ( ;; )
{
    ssize_t bytes_read = pread( fd, buffer,
        sizeof( buffer ), totalBytes + HEADER_SIZE );
    if ( bytes_read <= 0L )
    {
        break;
    }
    pwrite( fd, buffer, bytes_read, totalBytes );
    total_bytes += bytes_read;
}

ftruncate( fd, total_bytes );

close( fd );

You'll need to add the proper header files and some error checking.

Upvotes: 3

Jean-Fran&#231;ois Fabre
Jean-Fran&#231;ois Fabre

Reputation: 140226

In your case, you could

  • open the file in read-write
  • read char-by-char from your offset and write to the beginning, loop till the end of file (sounds suboptimal but you're on a RAM drive, and it's simple. Faster methods would imply read more bytes, maybe more complex to implement, and speed gain would have to be measured)
  • in the end you can truncate the file with truncate or ftruncate as explained here: How to truncate a file in C?

Upvotes: 1

Related Questions