CODEWITHSUNDEEP
apache-drill
user3897533
user3897533

Reputation: 477

Querying Last Column in a text file using apache-drill

I am unable to get the last column value Any suggestions, Selecting columns[2] or the last column name does not work

I have the text file as 

PolicyID~Name~AccountID
1~ABC Holdings~12
2~XYZ Insurance~2

When i do a Select * from dfs.root it works

When i do either AccountID or columns[2] i get an error.

How do i query the last column in drill?..I get the following error when i query the last column

org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IndexOutOfBoundsException: index: 32384, length: 4 (expected: range(0, 16384)) Fragment 0:0 [Error Id: dba982b9-0517-4ab2-a61f-dbf737c253da on 10.100.94.28:31010]

Upvotes: 1

Views: 358

Answers (1)

Paul Roesler
Paul Roesler

Reputation: 11

You will need to set up a new dfs configuration.

If you are using Drill on docker (https://drill.apache.org/docs/running-drill-on-docker/), here are the steps:

  1. Click on the Storage option at the top of the webpage, or go to this url: http://localhost:8047/storage
  2. Click on the update button on the "Enabled Storage Plugins" section
  3. Add this new configuration, after the csv configuration:
"txt": {
      "type": "text",
      "extensions": [
        "txt"
      ],
      "lineDelimiter": "\n",
      "fieldDelimiter": "~",
      "quote": "\"",
      "escape": "\"",
      "comment": "#",
      "extractHeader": true
    },
  1. Be sure to place your file in a location that drill can access and that it has a txt file extension (you may need to mount a folder from the host to the container e.g., /data/text/*), then query:
SELECT AccountID FROM dfs.`/data/text/*.txt`;

Upvotes: 1

Related Questions