CODEWITHSUNDEEP
scalaapache-sparkjdbc
A.b
A.b

Reputation: 27

Spark JDBC - How to stop automatic creation of table if table doesnt exist

I'm currently using jdbc to write data into an existing table.

jdbcDF.write
  .mode(Savemode.append)
  .jdbc("jdbc:postgresql:dbserver", "schema.tablename", connectionProperties)

I'm currently using code similar to above. When given a wrong table name, it will create that table and input that table there as opposed to throwing an error. Is there anyway to stop that feature.

Upvotes: 2

Views: 2270

Answers (1)

Michael Heil
Michael Heil

Reputation: 18505

This behaviour is not supported by Spark. Your would need to write your own logic around it.

According to the ScalaDocs on Enumeration SaveMode you have the following options when writing Data to a sink:

  • Append: Append mode means that when saving a DataFrame to a data source, if data/table already exists, contents of the DataFrame are expected to be appended to existing data.

  • ErrorIfExists: ErrorIfExists mode means that when saving a DataFrame to a data source, if data already exists, an exception is expected to be thrown.

  • Ignore: Ignore mode means that when saving a DataFrame to a data source, if data already exists, the save operation is expected to not save the contents of the DataFrame and to not change the existing data.

  • Overwrite: Overwrite mode means that when saving a DataFrame to a data source, if data/table already exists, existing data is expected to be overwritten by the contents of the DataFrame.

Upvotes: 2

Related Questions