CODEWITHSUNDEEP
clinux
user3586271
user3586271

Reputation: 41

error opening file for reading: Value too large for defined data type

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <iostream>
using namespace std;


#define FILEPATH "file.txt"
#define NUMINTS  (268435455)
#define FILESIZE (NUMINTS * sizeof(int))

int main()
{
    int i=sizeof(int);
    int fd;
    double *map;   //mmapped array of int's
  fd = open(FILEPATH, O_RDONLY);
    if (fd == -1) {
    perror("Error opening file for reading");
    exit(EXIT_FAILURE);
    }



    map = (double*)mmap(0, FILESIZE, PROT_READ, MAP_SHARED, fd, 0);
    if (map == MAP_FAILED) {
    close(fd);
    perror("Error mmapping the file");
    exit(EXIT_FAILURE);
    }

    for (i = 100000; i <=100100; ++i) {
    cout<<map[i]<<endl;
    }

    if (munmap(map, FILESIZE) == -1) {
    perror("Error un-mmapping the file");

    }
close(fd);
    return 0;
}

I am getting error that file size is too large.

Upvotes: 3

Views: 15662

Answers (2)

user1683793
user1683793

Reputation: 1313

I had this when I was trying to process a 2,626,351,763 byte file (which is more than fits in a signed 32 bit integer). The problem went away when I recompiling my program with cc -m64 (I am using the Sun C 5.13 SunOS_sparc 2014/10/20 compiler).

The 64 bit system is happy to deal with big (>2^32 byte) files but if the application is compiled in 32 bit mode, not so much.

Upvotes: 0

Basile Starynkevitch
Basile Starynkevitch

Reputation: 1

You should check that your mmap-ed file is large enough.

And make sure FILESIZE is a int64_t number (you need #include <stdint.h>):

#define FILESIZE ((int64_t)NUMINTS * sizeof(int))

Before your mmap call and after the successful open, add the following code, using fstat(2):

struct stat st={0};
if (fstat(fd, &st)) { perror("fstat"); exit (EXIT_FAILURE); };
if ((int64_t)st.st_size < FILESIZE) {
  fprintf(stderr, "file %s has size %lld but need %lld bytes\n",
          FILEPATH, (long long) st.st_size, (long long) FILESIZE);
  exit(EXIT_FAILURE);
}

At last, compile your program with g++ -Wall -g and use the gdb debugger. Also, strace(1) could be useful. And be sure that the file system for the current directory can deal with large files.

You may want or need to define _LARGEFILE64_SOURCE (and/or _GNU_SOURCE) e.g. compile with g++ -Wall -g -D_LARGEFILE64_SOURCE=1 ; see lseek64(3) & feature_test_macros(7)

addenda

Googling for 

Value too large for defined data type

gives quite quickly this coreutils FAQ with a detailed explanation. You probably should install a 64 bits Linux distribution (or at least recompile your coreutils appropriately configured, or use a different file system...)

Upvotes: 1

Related Questions