CODEWITHSUNDEEP
apache-sparkpyspark
N_A
N_A

Reputation: 101

How to use same table in two different spark sessions sessions?

If we have a big table. I created a DataFrame from it. In another spark session, I need same table. Is there any way we can use the same table to create dataframes in two different spark sessions?

Upvotes: 2

Views: 1307

Answers (2)

Matt Andruff
Matt Andruff

Reputation: 5155

Creating a hive table or writing the parquet/Orc files to a shared folder?

It is more permanent but it might make sense so one spark session ending can't kill the others data.

Upvotes: 0

vladsiv
vladsiv

Reputation: 2946

Take a look at createGlobalTempView it will be available across Spark Sessions.

If you want to have a temporary view that is shared among all sessions and keep alive until the Spark application terminates, you can create a global temporary view.

More information: https://spark.apache.org/docs/latest/sql-getting-started.html#global-temporary-view

Upvotes: 2

Related Questions