Reputation: 1760
I know I can get the version of Spark v2.2.1 that's running on Spark Master with this command:
http://<spark-master>:4040/api/v1/version
which will return something like
{
"spark" : "2.2.1"
}
However, I also want to check the version of Spark running on each Worker. I know I can get a list of all Workers thusly:
http://<spark-master>:8080/json/
which will return a response similar to
{
"url": "spark://<spark-master>:7077",
"workers": [{
"id": "worker-20180228071440-<ip-address>-7078",
"host": "<ip-address>",
"port": 7078,
"webuiaddress": "http://<ip-address>:8081",
"cores": 8,
"coresused": 8,
"coresfree": 0,
"memory": 40960,
"memoryused": 35875,
"memoryfree": 5085,
"state": "ALIVE",
"lastheartbeat": 1519932580686
}, ...
],
"cores": 32,
"coresused": 32,
"memory": 163840,
"memoryused": 143500,
"activeapps": [{
"starttime": 1519830260440,
"id": "app-20180228070420-0000",
"name": "<spark-app-name>",
"user": "<spark-app-user>",
"memoryperslave": 35875,
"submitdate": "Wed Feb 28 07:04:20 PST 2018",
"state": "RUNNING",
"duration": 102328434
}
],
"completedapps": [],
"activedrivers": [],
"status": "ALIVE"
}
I'd like to use that information to query each Spark Worker's version. Is this possible?
Upvotes: 0
Views: 702