CODEWITHSUNDEEP
apache-sparkhadoophiveinformaticainformatica-cloud
Dozel
Dozel

Reputation: 159

Informatica load data into hive partition tables error

Previously, we had this error:

Caused by: java.util.concurrent.ExecutionException: org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Number of dynamic partitions created is 1020, which is more than 1000.

we set hive.exec.max.dynamic.partitions to 1500, and it solved the problem.

But this time, same infa mapping giving us this error

Caused by: java.util.concurrent.ExecutionException: org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Exception when loading 1020 in table table_name with loadPath=s3://bucket/table_name/.hive-staging_hive_2024-05-02_08-59-01_073_6154768442829892392-1/-ext-10000;
    at com.informatica.bootstrap.Async$.flush(async.scala:87)
    at com.informatica.exec.InfaSpark0$.main(InfaSpark0.scala:62)
    at com.informatica.exec.InfaSpark0.main(InfaSpark0.scala)

Upvotes: 0

Views: 43

Answers (0)

Related Questions