CODEWITHSUNDEEP
c++hadoop
L.S
L.S

Reputation: 179

Read and print text file from HDFS

I have this short code that read bytes from a text file on the Hadoop File System (HDFS) using libhdfs. It compiles and works fine. I am now trying to change the code so I could read a the content of the text file as well.

The following is the code I have right now for printing a text file from HDFS:

#include "jni.h"
#include "hdfs.h"
#include "string.h"
#include "stdlib.h"

int
main(int argc, char **argv)
{
  int MAXBUFLEN = 1024;

  hdfsFS fs = hdfsConnect("default", 0);
  const char* readPath = "/tmp/testfile.txt";
  hdfsFile readFile = hdfsOpenFile(fs, readPath, O_RDONLY, 0, 0, 0);
  if(!readFile) {
    fprintf(stderr, "Failed to open %s for writing!\n", readPath);
    exit(-1);
  }
  char buffer[MAXBUFLEN+1];

  int bytes = hdfsRead(fs, readFile, buffer, strlen(buffer));
  buffer[MAXBUFLEN] = '\0';
  hdfsCloseFile(fs, readFile);
  return 0;
}

Upvotes: 1

Views: 1623

Answers (1)

p-a-o-l-o
p-a-o-l-o

Reputation: 10047

You should initialize your buffer this way:

char buffer[MAXBUFLEN+1] = {};

then pass the maximum buffer length (don't use strlen) to the read function:

int bytes = hdfsRead(fs, readFile, buffer, MAXBUFLEN);

No need of this line:

buffer[MAXBUFLEN] = '\0';

since the buffer is properly initialized. You can then output the whole buffer as a c string:

std::cout << buffer << std::endl;

The string length, now, should be equal to the read bytes:

assert(strlen(buffer)==bytes);

Upvotes: 1

Related Questions