Variables using parser.add_argument or importing from a CSV file

Question

I am working on an API script that gets the variables from parser arguments (--argument <>) in the command. What I want to do is if there is no variable given from the command line, to import the variables from a CSV file instead.

The way that I'm assuming to do this is using or after my variables such as: token = args.token or (csv value)

There are many variables that need to be imported from this CSV file, how would I load a file.csv such as hostname,token,username,password,..etc into the variables in that or statement such as: token = args.token or (csv value)?

Then, how would I loop my script for each new line of the CSV file, with each new variable? Any help would be appreciated!

Sample data: (parameters.py)

parser = argparse.ArgumentParser(description = 'Variables')
parser.add_argument('--token', help = "API token")
parser.add_argument('--hostname', help = "Hostname of client")
parser.add_argument('--ip', help = "IP address")
args = parser.parse_args(sys.argv[1:])
token = args.token or (csv import)
hostname = args.hostname or (csv import)
ip = args.ip or (csv import)

Sample data: (main.py)

headers = {
    "Authorization": f'Token {parameters.token}',
    'Content-type':'application/json', 
    'Accept':'application/json'
    }

payload = {
  "name": str(parameters.hostname),
  "custom_fields": {}
}

if parameters.ip:
    payload['custom_fields']['management_ip_address'] = str(parameters.ip)

register = requests.post(
    'https://url',
    headers=headers, timeout=60, verify=False, data=json.dumps(payload))
regstatus = register.status_code
if regstatus == 201:
    print (f'Task completed for {parameters.hostname}.')

Sample data: file.csv

token, hostname1, ip
token, hostname2, ip

Sample output:

Task completed for hostname1.
Task completed for hostname2.

Answer 1

I've dealt with this kind of problem recently, "config-file values unless command-line values", so I've got a solution already to go.

My thinking is that your command-line args take precedence, so even if your user (just you?) entered an argument in both places, the command-line value wins.

I mocked up this sample config CSV:

hostname,token,username,password
Hostname (csv),Token (csv),Username (csv),Password (CSV)

Instead of looping over every possible arg and checking if its CSV counterpart exists, we can leverage one aspect of the csv module and two aspects of the argparse module.

First the csv module.

1. we can read a CSV row of config values in as a dict with the DictReader, and end up with a dict like this:

   reader = csv.DictReader(f)
   config_row = next(reader)
   # {'hostname': 'Hostname (csv)', 'token': 'Token (csv)', 'username': 'Username (csv)', 'password': 'Password (CSV)'}
   

And onto the argparse module, and specifically the Namespace class which is what we get when we callparse_args():

1. A Namespace holds all the positional and optional arguments that were found during parsing. We can create a Namespace ourselves by just passing in a dict with keys that match the names of the arguments we want. Here I’m using the dict from the previous CSV reader:

   csv_args = argparse.Namespace(**config_row)
   # Namespace(hostname='Hostname (csv)', password='Password (CSV)', token='Token (csv)', username='Username (csv)')
   

2. We can combine Namespaces, and there’s a clear order of precedence. We call parse_args(namespace=csv_args) to parse the command-line and fill-in any missing args with the CSV values (you don’t need to specify sys.argv[1:], parse_args knows what to do):

args = parser.parse_args(namespace=csv_args)
# Namespace(config_file='config.csv', hostname='Host (cmdline)', password='Password (CSV)', token='Token (csv)', username='Username (csv)')

Here's the whole script. I’m calling parse_args twice. Once to check for and read the CSV, and finally to take the command-line values:

#!/usr/bin/env python3
import argparse
import csv

parser = argparse.ArgumentParser()

parser.add_argument('--hostname')
parser.add_argument('--token')
parser.add_argument('--username')
parser.add_argument('--password')
parser.add_argument('--config-file', help='a CSV file with a row of any of the above arguments as their own columns')

pre_args = parser.parse_args()

csv_args = argparse.Namespace()
if pre_args.config_file:
    with open(pre_args.config_file, newline='') as f:
        reader = csv.DictReader(f)
        config_row = next(reader)
        print(config_row)
    
    csv_args = argparse.Namespace(**config_row)
    print(csv_args)

args = parser.parse_args(namespace=csv_args)

print(args)

When I run the following command:

./main.py --hostname 'Host (cmdline)' --config-file config.csv

I get:

Namespace(config_file='config.csv', hostname='Host (cmdline)', password='Password (CSV)', token='Token (csv)', username='Username (csv)')