Anubhav Sisodia
Anubhav Sisodia

Reputation: 1

Bulk Load of binary data in postgreSQL using pgloader

I want to load a bulk data in postgreSQL DB using pgloader. For the same I have created a load file (loadfile.txt) as:

LOAD CSV
    FROM '/tmp/data.csv'  -- Path to your CSV file
    INTO postgresql://username:password@localhost:5432/mydb?sslmode=require
    WITH truncate,
         fields terminated by '|',
         fields optionally enclosed by '"'
    SET client_encoding to 'utf8'
    BEFORE LOAD DO
        $$ DROP TABLE IF EXISTS bills_table; $$,
        $$ CREATE TABLE bills_table (
            id TEXT,
            geocode TEXT,
            xml_bill_obj BYTEA
        ); $$
    CAST
        column id to text,
        column geocode to text,
        column xml_bill_obj to bytea using bytea
    ;

My data.csv file is have input feed with 3 fields delimited by a pipe(|) and thousands of records. Data file is having first two fields as plain text and third field is created with the basic compress method of a PERL program.

This compressed data is actually an XML (with 200 lines of records) extracted from a bigger dataset.

As my postgres is installed on linux I'm running this as:

./pgloader loadfile.txt

I executed this but it is giving me error at CAST for an syntax error.

[app-vm-01:/opt/app/]$ ./pgloader loadfile.txt
2024-08-02T16:24:40.036001Z LOG pgloader version "3.6.7~devel"
KABOOM!
ESRAP-PARSE-ERROR: At
 
          ); $$
  CAST type binary TO byte
  ^ (Line 15, Column 0, Position 712)
 
In context COMMAND:
 
While parsing COMMAND. Expected:
 
     the character Tab
  or the character Newline
  or the character Return
  or the character Space
  or the character , (COMMA)
  or the string "--"
  or the string "/*"
  or the character ; (SEMICOLON)
  or the string "after"
  or the string "before"
  or the string "do"
  or the string "execute"
  or the string "set"
  or the string "with"
An unhandled error condition has been signalled: At

Upvotes: 0

Views: 90

Answers (0)

Related Questions