CODEWITHSUNDEEP
apache-sparkapache-spark-sqlhiveql
Dmitry Petrov
Dmitry Petrov

Reputation: 1547

Create a "temporary" table in spark-SQL, not spark-scala?

I have loaded a table from an input file.

CREATE TABLE MyTable (
    ID INT,
    VALUE FLOAT,
    RATE  INT
...

LOAD DATA LOCAL INPATH 'MYPATH' INTO TABLE MyTable;

Now I'd like to create a new based on this one

DerivedTable = 
    SELECT ID, VALUE*RATE AS Total
    FROM MyTable
    WHERE VALUE IS NOT NULL;

Then I'm going to use this table as a source for other tables and for outputs.

What is a correct Sql (or Hive) way to create this "temporary" table? This should work in spark-sql?

PS: I know how to do that in spark-shell. But that is not what I'm looking for.

Upvotes: 0

Views: 12777

Answers (1)

Alper t. Turker
Alper t. Turker

Reputation: 35229

You can:

CREATE TEMPORARY VIEW DerivedTable AS (
   SELECT ID, VALUE*RATE AS Total
   FROM MyTable
   WHERE VALUE IS NOT NULL);

Upvotes: 6

Related Questions