CODEWITHSUNDEEP
pythonjsonargparse
Estrobelai
Estrobelai

Reputation: 309

Python: From arg parser to dictionary

I have a file in Python 3.8 called "stores.py" This file has a method called "scan_transactions" which takes 2 positional arguments: "store" and "checkpoint". The method basically scans for store transactions in a PostgreSQL table by making use of REGEX patterns. When the code reaches the last transaction id for that particular store in the transactions table, another table (checkpoint table) is then used and updated to indicate what is the latest max transaction id for any given store.

At the moment I am passing both arguments from a pre-defined dictionary similar to the one below:

dict_stores = {'store1': 'checkpoint_store1', 'store2': 'checkpoint_store2','store3': 'checkpoint_store3'}

Currently the code is looking like the below:

def store_transactions(store: str, checkpoint_name: str)
.
.
.
.
.

if __name__ == '__main__':

    for store, checkpoint in shops.dict_stores.items():
        LOG.debug(f'Processing store : {store}, checkpoint: {checkpoint}')
        store_transactions(store, checkpoint)

I am now looking to make this more dynamic and allow the user to pass the stores they want to process the transactions for as a batch job right before they execute it. This would be using the command line below:

"stores.py" --stores -store1 -store2 -store3...etc.

The command above would then replace this pre-fixed dictionary and create a dictionary dynamically. Does anyone know how I could use "arg parser" to somehow programatically convert the arguments "-shop 1", "-shop2" into a dictionary like the one above (with their respective checkpoints as values) and process all stores with the same loop I am currently running with?

Upvotes: 0

Views: 374

Answers (3)

JL Peyret
JL Peyret

Reputation: 12164

Note, I think you need to use positional argparse arguments to have them repeat (i.e. you dont have --store option names for them). Or maybe I am confusing with optparse, using mostly Click these days.

The nargs part of the doc covers that so it looks like you could use --store too. Not super clear with examples. That said, that's more typing for the users, so I'd go with positionals.

import argparse

#the existing dictionary
lookup = {'store1': 'checkpoint_store1', 'store2': 'checkpoint_store2','store3': 'checkpoint_store3'}

#from doc @ https://docs.python.org/3/library/argparse.html#example
parser = argparse.ArgumentParser(description='Process some stores.')

#Option 1 your loop checks for valid stores
# parser.add_argument('stores', type=str, nargs='+', help='stores')

#Option2 argparse checks for valid stores
parser.add_argument('stores', type=str, nargs='+', help='stores', choices=lookup.keys())

args = parser.parse_args()
user_stores = args.stores
dict_stores = {}

#check in loop
for store in user_stores:
    try:
        dict_stores[store] = lookup[store]
    #pragma: no cover pylint: disable=unused-variable
    except (KeyError,) as e: 
        print(f" unknown store {store}. known : {' '.join(lookup.keys())}")

# if you use argparse to check this can be simplified to
# dict_stores[store] = {store: lookup[store] for store in user_stores}


print(f"{dict_stores}")

output:

(venv38) me@explore$ py test_301_arg.py store1 store2
{'store1': 'checkpoint_store1', 'store2': 'checkpoint_store2'}


(venv38) me@explore$ py test_301_arg.py store1 store4
usage: test_301_arg.py [-h] {store1,store2,store3} [{store1,store2,store3} ...]
test_301_arg.py: error: argument stores: invalid choice: 'store4' (choose from 'store1', 'store2', 'store3')


(venv38) me@explore$ py test_301_arg.py --help
usage: test_301_arg.py [-h] {store1,store2,store3} [{store1,store2,store3} ...]

Process some stores.

positional arguments:
  {store1,store2,store3}
                        stores

optional arguments:
  -h, --help            show this help message and exit

Upvotes: 1

ti7
ti7

Reputation: 18796

Alternate solution: simply read a JSON configuration file, perhaps making the argument either the filename or a trigger for stdin

import json

    ...
    with open(path_config) as fh:
        config = json.load(fh)  # config is now a Python dict

Upvotes: 0

ti7
ti7

Reputation: 18796

I've found it convenient to use a structure which splits on , to read a list of possibilities

def parse_args(args_external=None):
    """ build an arguments object with required attributes from user input """

    parser = argparse.ArgumentParser(
        description="Example Program",
    )

    parser.add_argument(
        "--foo",
        default="",
        help="comma-separated collection of bars")

    arguments = parser.parse_args(args_external)  # enables automated testing (otherwise None -> sys.argv)

    _foo = []
    for bar in arguments.foo.split(","):
        if not bar:  # allow skipping ,,
            continue
        validatebar(bar)  # sys.exit with message if invalid
    arguments.foo = _foo  # clobber the original reference

this is consumed like

python3 ./myscript.py --foo bar1,bar2,bar3

Upvotes: 1

Related Questions