CODEWITHSUNDEEP
databricks
Siddhant Jain
Siddhant Jain

Reputation: 549

Fetching username inside notebook in Databricks on high concurrency cluster?

While trying to fetch user data on high concurrency cluster, I am facing this issue. I am using the command below to fetch the user details

dbutils.notebook.entry_point.getDbutils().notebook().getContext().tags().apply('user')

Below is the error log, for the run. Any help would be really appreciated.

Py4JError: An error occurred while calling o475.tags. Trace:
py4j.security.Py4JSecurityException: Method public scala.collection.immutable.Map com.databricks.backend.common.rpc.CommandContext.tags() is not whitelisted on class class com.databricks.backend.common.rpc.CommandContext
    at py4j.security.WhitelistingPy4JSecurityManager.checkCall(WhitelistingPy4JSecurityManager.java:409)
    at py4j.Gateway.invoke(Gateway.java:294)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:251)
    at java.lang.Thread.run(Thread.java:748)

Upvotes: 13

Views: 16167

Answers (4)

Arkady
Arkady

Reputation: 15069

I've been using this:

user_id = spark.sql('select current_user() as user').collect()[0]['user']

current_user() is a documented SQL function in Databricks

Upvotes: 8

Karthikeyan Rasipalay Durairaj
Karthikeyan Rasipalay Durairaj

Reputation: 2338

You can use below code :

import json

# parse x:
y = dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson() 
res = json.loads(y)
print(res['tags']['user'])

Note : Tested code enter image description here

Upvotes: 4

Federico Raimondi Cominesi
Federico Raimondi Cominesi

Reputation: 796

You can retrieve the information by using dbutils command:

dbutils.notebook.entry_point.getDbutils().notebook().getContext().userName().get()

Upvotes: 27

Ben Corcoran
Ben Corcoran

Reputation: 69

This is a gross work around, but I haven't found much better yet.

import uuid
import shutil

# Create a unique temporary table location
tmpTable = f'/tmp/identifier/{uuid.uuid4()}'

# Write a single line to a delta format table.
spark.range(1).write.format('delta').save(tmpTable)

# Extract the username from the delta history 
username = spark.sql(f'DESCRIBE HISTORY delta.`{tmpTable}`').select('userName').collect()[0]['userName']

# Delete the temporary table
shutil.rmtree('/dbfs'+tmpTable)

Upvotes: -1

Related Questions