Reputation: 11
I have one python script with it's settings in one seperate json config file. The json file looks like this:
{
"connection" : {
"db_server" : "server",
"db_name" : "table1",
"db_user" : "user1",
}}
Now I need to run the same python file more than one time, each with other settings in the config file. The other settings would look like this:
{
"connection" : {
"db_server" : "server",
"db_name" : "table2",
"db_user" : "user2",
}}
I don't need to change anything in the Python script. I open the json file in my Python script like this:
with open('settings.json') as json_data_file:
data = json.load(json_data_file)
json_data_file.close()
Since you cannot add comments in a json file, I don't know the easiest way to do this. I want the Python script to run simultaneously two times, each time with other settings for the json file.
Thanks in advance!
Upvotes: 1
Views: 2199
Reputation: 128
A simple solution is a new script that parses your JSON file, imports your Python scripts, and then executes that scripts with different parameters using concurrent.
An example (adapted from the example for ThreadPoolExecutor):
import concurrent.futures
import json
from YourModule import MainFunction
# First, open and load your JSON file
parameter_dict = json.load(open('yourfilename.json'))
# Do some parsing to your parameters in order
# (In your case, servers, tables, and users)
parameters_to_submit = [[entry['db_server'], entry['db_table'], entry['db_user'] for entry in parameter_dict.values()]
# Now, generate a ThreadPool to run your script multiple times
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
# Submit the function + parameters to the executor
submitted_runs = {
executor.submit(MainFunction, params[0], params[1], params[2]): params
for params in parameters_to_submit
}
# as the results come in, print the results
for future in concurrent.futures.as_completed(submitted_runs):
url = submitted_runs[future]
try:
data = future.result()
except Exception as exc:
print(f'Script generated an exception: {exc}')
else:
# if need be, you could also write this data to a file here
print(f'Produced result {data}')
Upvotes: 0
Reputation: 965
sys.argv[1]
) to choose which config file to use. I personally recommend this approach.Upvotes: 3