CODEWITHSUNDEEP
apache-sparkdatabrickscluster-computingservice-principal
Vzzarr
Vzzarr

Reputation: 5700

Unable to assign Databricks Singe User Cluster to a Service Principal

From the Databricks UI when I try to assign a single user to access the cluster it seems like I can only pick human users and no possibility to pick Service Principals:

How to assign a Service Principal to Single user access my Databricks Cluster?

Upvotes: 2

Views: 313

Answers (3)

TA1992
TA1992

Reputation: 9

Following on from Kashyap's anwser, i neededed to include the following to the json request

  "spark_conf": {
     "spark.databricks.cluster.profile":"singleNode",
     "spark.master":"local[*, 4]"
  },
 "custom_tags": {
  "ResourceClass":"SingleNode"
}

in order to be able to interact with tables and not present warning messages in the UI.

Upvotes: 0

Kashyap
Kashyap

Reputation: 17524

Can't be done using WebUI.

Use CLI.

You can get:

  • the SP's id (single_user_name value) using databricks service-principals list
  • spark versions: databricks clusters spark-versions
  • node types: databricks api get /api/2.0/clusters/list-node-types
databricks clusters create --json '
{
    "cluster_name": "my-cluster",
    "spark_version": "13.3.x-scala2.12",
    "node_type_id": "i3.xlarge",
    "driver_node_type_id": "i3.xlarge",
    "single_user_name": "9999b04c-890c-0000-1111-be334c3406e8",
    "data_security_mode": "SINGLE_USER",
    "runtime_engine": "STANDARD",
    "num_workers": 0
}
'

Upvotes: 3

Vzzarr
Vzzarr

Reputation: 5700

Based on this post: https://community.databricks.com/t5/data-engineering/databricks-single-user-cluster-is-not-able-to-assign-service/td-p/4592

I found out that is possible to assign a Service Principal to Single user access via CLI:

databricks clusters edit <Cluster ID> <Spark Version> 
  --cluster-name "<Cluster Name>" --data-security-mode SINGLE_USER 
  --single-user-name <Service Principal Client ID> --num-workers 1 
  --node-type-id <Node Type> --no-wait

In order to get the right Spark Version: https://stackoverflow.com/a/77102743/4725074

In order to assign my token locally: https://stackoverflow.com/a/63519211/4725074

In order to get the correct Node Type: https://docs.databricks.com/api/gcp/workspace/clusters/listnodetypes

Upvotes: 0

Related Questions