CODEWITHSUNDEEP
pythonsubprocessdatabricksdatabricks-connect
vaibhav
vaibhav

Reputation: 85

Can I get the exact databricks job link url while running the databricks job?

I am trying to get the exact databricks job run link url while running the databricks job. Below is my setup and what I want to achieve.

How I am triggering databricks job :

submit_body = {'run_name': 'run_name',
'libraries': [],
'spark_python_task': {
    'python_file': 'python file location',
    'parameters': [
        '--databricks-job-id', '{{job_id}}',
        '--databricks-run-id', '{{run_id}}',
        '--databricks-host', 'host',
        '--other-params', '...'

    ]
}}
cmd = ["databricks", "runs", "submit", "--version", "2.1", "--json", json.dumps(submit_body), "--wait"]
cmd_out = subprocess.run(cmd, capture_output=True)

Here I can show user the job run page : f"{host}#job/runs", but I want to show them the exact job link i.e. : f"{host}#job/{job_id}/run/{run_id}".

I can extract the job id and run id once the job is complete but I want to show the exact job link url while the job is running. Also, i cannot remove --wait from the command as some further preprocessing happens after the job.

Does someone know a way to achieve this ?

Upvotes: 2

Views: 222

Answers (0)

Related Questions