CODEWITHSUNDEEP
hadoophiveteradatasqoop
paolov
paolov

Reputation: 2327

Why does sqoop map DECIMAL types in an import from Teradata to DOUBLE?

I've got Sqoop version: 1.4.6

When importing data from Teradata into Hive, I find that DECIMAL types are converted to DOUBLE.

I wonder why that is when Hive has an exact equivalent datatype.

Upvotes: 0

Views: 1996

Answers (1)

Dev
Dev

Reputation: 13753

This is because they wrote it generically for all the RDBMS.

DECIMAL is mapped with DOUBLE.

Useful part of source code:

 public static String toHiveType(int sqlType) {

      switch (sqlType) {
          case Types.INTEGER:
          case Types.SMALLINT:
              return "INT";
          case Types.VARCHAR:
          case Types.CHAR:
          case Types.LONGVARCHAR:
          case Types.NVARCHAR:
          case Types.NCHAR:
          case Types.LONGNVARCHAR:
          case Types.DATE:
          case Types.TIME:
          case Types.TIMESTAMP:
          case Types.CLOB:
              return "STRING";
          case Types.NUMERIC:
          case Types.DECIMAL:
          case Types.FLOAT:
          case Types.DOUBLE:
          case Types.REAL:
              return "DOUBLE";
          case Types.BIT:
          case Types.BOOLEAN:
              return "BOOLEAN";
          case Types.TINYINT:
              return "TINYINT";
          case Types.BIGINT:
              return "BIGINT";
          default:
        // TODO(aaron): Support BINARY, VARBINARY, LONGVARBINARY, DISTINCT,
        // BLOB, ARRAY, STRUCT, REF, JAVA_OBJECT.
        return null;
      }
  }

They also put warning for this mapping for DATE, TIME, TIMESTAMP, DECIMAL, NUMERIC SQL data types.

  if (HiveTypes.isHiveTypeImprovised(colType)) {
    LOG.warn(
        "Column " + col + " had to be cast to a less precise type in Hive");
  }

Check source code for details.

Upvotes: 2

Related Questions