How can we have table grouping in Spark the same way as SQL Server?

Question

In SQL Server we have a database, inside database we have schema and under that schema we have table, like DatabaseName.SchemaName.TableName. And we also refer to Database as Catalog in SQL Server.

But in Spark, Catalog means an interface that we use to interact with Spark. I also noticed same result when I typed

• spark.sql('show databases').show() and
• spark.sql('show schemas').show()

Does database and schema mean same thing in Spark? If not what's the difference? In Spark, is it possible to have it the same way like SQL server has DatabaseName.Schema.TableName?

Answer 1

From the docs

Please note that the usage of SCHEMAS and DATABASES are interchangeable and mean the same thing.

SHOW DATABASES

And CREATE TABLE supports only two-part names.