CODEWITHSUNDEEP
apache-sparkenvironment-variablesspark-webui
sue
sue

Reputation: 43

Hide Spark environment variable value from ps and web-ui

I am using Spark 2.3.1 on Mac, in Java.

I have confidential security info stored in an environment variable. However, as it's confidential, I don't want to expose its value through ps -e nor from http://localhost:4040/environment/.

Is there a way within Spark for me to hide the value please? Or anyway by code seal the value, while not affecting other Spark/Java functions.

Upvotes: 0

Views: 409

Answers (1)

Arnon Rotem-Gal-Oz
Arnon Rotem-Gal-Oz

Reputation: 25909

You shouldn't use environment variables to store confidential information there are tools like Hashicorp Vault

regarding ps -e - if someone has access to the machine they can echo the environment variable anyway. As for the spark UI - you can secure the access to it see here

Upvotes: 0

Related Questions