Understanding positional arguments in Python

Question

For the following Python script:

from sys import argv

script, input_encoding, error = argv

def main(language_file, encoding, errors):

    line = language_file.readline()

    if line:

        print_line(line, encoding, errors)
        return main(language_file, encoding, errors)

def print_line(line, encoding, errors):
    next_lang = line.strip()
    raw_bytes = next_lang.encode(encoding, errors=errors)
    cooked_string = raw_bytes.decode(encoding, errors=errors)

    print(raw_bytes, "<===>", cooked_string)

languages = open("languages.txt", encoding="utf-8")

main(languages, input_encoding, error)

Looking at the main function I do not understand the following line:

print_line(line, encoding, errors)

Why are we calling the print_line function and passing arguments to it that are named exactly the same as its parameters?

print_line()

When I attempt to call the print_line() argument without passing arguments Python is outputting:

print_line() missing 3 required positional arguments: 'line', 'encoding', and 'errors'

Answer 1

OP: print_line() missing 3 required positional arguments: 'line', 'encoding', and 'errors'

The error is obvious since it is the way the function print_line() was defined.

Furthermore:

def print_line(line, encoding, errors):
    print(line, encoding, errors)


line = 1
encoding = 2
errors = 3

print_line(errors, encoding, line)

OUTPUT:

3 2 1

Note: It is positional, not naming arguments

EDIT: 1

def abc(a,b,c=2):

return a+b+c

abc(1,2) #both positional argument and c is default

5

abc(2, b=3) # positional, named and again c is default

7

abc(a=2,b=4) # both named argument and c is default

8

EDIT 2:

OP: What is the purpose of a positional argument please?

Well ..

Short answer: A positional argument is any argument that's not supplied as a key=value pair.

To understand what that means, unfortunately, is somewhat involved.

The term "argument" is used somewhat imprecisely throughout the programming community and especially in Python documentation.

Technically arguments are what you pass into functions and parameters are what you define as the names/placeholders for those arguments.

So, when I define a function thus:

def foo(a,b):
    return a+b

... and call it like so:

foo(1,3)

... then a and b are my parameters while 1 and 3 are arguments for a given call to that function.

Now this is a quibble. People will often refer to a and b as "arguments" to their function when they are actually the names (parameters) which will contain the arguments while the function is executing.

Now, with this point made, understand that Python supports four classes of parameters: "required positional" (the kind you've seen for just about any other programming language), "optional" ... or "defaulted" ... positional parameters with some default value specified, "star args" (similar to the VARARGS support in some other languages such as C/C++ and Java), and "kwargs."

The latter is almost unique to Python (though Ruby has very similar support).

So you can define a function with a parameter list like:

def bar(a, b, c=None, d=[], *e,  **opts):
    '''bar takes all sorts of arguments
    '''
    results = dict()
    results['positional 1'] = a
    results['positional 2'] = b
    results['sum']=a+b
    results['optional'] = list()
    for each in e:
        results['optional'].append(each)
    if c is not None:
        results['default over-ridden']=c
    else:
        results['default']='no arg supplied for parameter c'
    d.append(results)
    if 'verbose' in opts and opts['verbose']:
        for k,v in results:
            print '%s:%s' % (k, v)
    return (results, d)

... this, rather contrived, example has a couple of normal, traditional positional parameters (a and b), and optional third and fourth parameters (one of which will default to the special Python singleton value None if bar() is called with only two arguments and the other which will default to a list) as well as an optional variable number of additional arguments (which are passed into our function through the parameter named "e") and, finally, bar() can accept any number of "keyword" arguments (passed to it as key-value pairs and referenced through the parameter "opts" (in my example).

There's a lot to digest there. First there are a and b. Those are just like you'd see in most programming languages. If you call this with only one argument you'll get an error because the function requires two arguments. Then there are c and d ... these parameters can be supplied with arguments ... but if the function is called with only the two required arguments than parameter c will refer to "None" and d will refer to the list ... which was instantiated at the time we defined the function!

Whoa! Re-read that last bit because it's a common source of confusion to people who make the mistake of defining functions with parameters that default to mutable types (lists, dictionaries, sets, or most instances of your custom defined classes).

When you're defining a function in Python the interpreter is executing code. It's executing the def statement and evaluating the arguments (which become your function's parameters). So a Python virtual machine instantiates a list (the [] --- empty list literal) as the function is defined. The parameter (e in my example) is now bound to that list, just as any Python "variable" (name) is bound to any other object. And the object to which it refers will be accessible any time bar() is called with three or fewer arguments.

Here's the tricky bit: any time you call bar with more than three arguments then the parameter e will be bound (for the duration of that particular call) to the fourth argument. The underlying list will be hidden for the duration of that call. Such the default list object is contained in a closure and would normally be completely inaccessible outside of the function. (In this example I've including a reference to it in my return statement, showing how we can export a reference to that enclosed object if we choose).

So, if I pass bar() four arguments then it will attempt to modify the object referred to through its "e" parameter using that object's 'append' method. (Obviously that will fail and raise an exception for any object that doesn't support an append method). Every time I call bar() with only three arguments then I'm modifying that enclosed list object within its closure.

This can be useful but is rarely what new programmers expect when they are learning Python. So, as a rule, don't define functions with mutable objects as defaults to your parameters. When you understand the semantics well enough then you know when to break that rule.

I used "None" as my default for the other parameter for a reason. It's often useful to use "None" as the default for any parameter for which you want to have a default value and, then, to explicitly test "is None" and supply your own default from within the body of your function. In this way you can distinguish between your default parameter value and any argument that was explicitly passed to you (and happened to match your default). (This will also prevent the case where you might inadvertently create a mutable closure as previously described. The assignments/bindings occurring in the body of your function will result in a new instantiation for every call that reaches that condition).

So we've covered the required positional parameters, and those which supply defaults (and are, thus, optionally positional parameters).

With parameter 'e' we see Python's support for variable numbers of arguments. Any arguments specified after the fourth will all be gathered up into a tuple and passed to us through our parameter 'e' ... except for any arguments of the form: this=that.

Which brings us, at last. to "opts." Conventionally Python programmers will refer to this parameter as "kwargs" (key/word arguments). I named it "opts" to emphasize that point that "kwargs" is only a conventional bit of terminology and, technically, a bit confusing since, as I've belabored throughout this text, it's a parameter which is referring to any keyword arguments which might have been passed as parameters through some function call.

It's possible to write all your functions such that they take NO positional arguments and are only defined with a "key/words arguments" parameter. You can then ensure that callers to your function must always spell out which argument is bound to what name every time that call on you. This can be handy if your function might have catastrophic consequences in any case where it was called with arguments in the wrong order.

I realize this is confusing ... and I definitely recommend that you play with different function definitions and a wide variety of calls to see how these interact.

I'll also note one additional "gotchya" that can bite you. Your callers CANNOT pass kwargs (opts) values with keys that have names conflicting with your parameter names. Try calling bar() with an argument list like: bar(1,2,3,4,a=99) and you'll get an exception like this: "TypeError: bar() got multiple values for keyword argument 'a'"

Python is parsing the arguments into parameters by managing a namespace (like a dictionary). Attempting to supply a key/word argument which any keys that match your parameter names creates an ambiguity ... and thus raises an exception.

I'll also add two additional notes to this already cumbersome answer.

When you're calling functions in Python you can pass arguments like so:

myargs=(1,2,3)
bar(*myargs)

... and this will be treated as though you'd called it with bar(1,2,3) --- or as if you make the following function call: apply(bar, myargs).

In fact it used to be that you had to use the apply() function in order to accomplish this layer of indirection (back when you could define functions with *foo parameters but not call them with *foo arguments).

(The addition of the *args syntax on function calls largely displaced the use of Python's apply() builtin function).

... and lastly it's possible to pass a dictionary of kwargs using this syntax:

mykwargs={'z':99, 'whatever':'yikes'}
bar(1,2,3, **mykwargs)

... or combined with my previous example:

bar(*myargs, **mykwargs)

So it's vital to understand this distinction between what * and ** mean when defining functions and what they mean when calling them. The meanings are complementary to one another and intuitive if you understand the distinction between arguments and parameters (despite how the term "arguments" is more commonly used).

Answer 2

Why are we calling the print_line function

Well, to execute it obviously ;-)

and passing arguments to it

Because the function is defined to take arguments - it cannot do its job if you don't give what it needs to work on.

that are named exactly the same as its parameters?

This is actually totally irrelevant - it just happens that the variables in main() are named the same as the function's arguments (which makes sense since we're taling about the same things - a "line of text", an encoding name and a value that describes how to handle encoding errors), but you pass litteral (unnamed) values or variables with just any arbitrary name, it would work just the same.

A function is a (mostly) self-contained unit of work. It usually uses some internal ("local") variables which only it can sees and that only exists during the function's execution. It also usually takes arguments: values that must be passed by the caller, and are bound to the matching arguments names (in our case, the first value will be bound to the line name, the second to the encoding name etc). The local names (local variables and arguments) are totally unrelated to the name under which those values are known in the caller's scope (if even they are bound to names - as I said you can as well pass literal values or other anonymous objects)

When I attempt to call the print_line() argument without passing arguments Python is outputting "print_line() missing 3 required positional arguments: 'line', 'encoding', and 'errors'"

Yes, of course. The function needs three arguments, so you have to pass three arguments, plain and simple. The fact that you have three local variables by the same name in the caller scope (the main function) will not automagically "fill in" those arguments for you, and the print_line function knows absolutely nothing about it's caller's scope anyway.

Note that the terms "positional" and "named" arguments mostly to refer to how you pass the arguments themselves - by position (the default), which I already explained above, or by name (ie print_line(line="hello", errors="ignore", encoding="utf-8"), which allow you to pass the arguments in a different order, or from a dictionnary you've built dynamically etc etc, but you first need to understand the concepts of function, arguments and scope before going further...

I strongly suggest that you do the official tutorial, which does have a chapter on functions - it's mostly intended at peoples with already some programming experience (it's not a CS101 course) but it's still well explained.

Answer 3

The function requires three arguments, and normally, you provide them in the order specified.

Python allows you to pass them in any order you like, though:

print_line(encoding='ascii', line='hello', errors=None)

or even

my_dict = {
    'line': 'privet, mir!',
    'errors': None,
    'encoding': 'utf-8'
}
print_line(**my_dict)

However, the function requires all these arguments; you have to pass them in somehow, or you get an error message, like you well noticed.

The variables' scope is local: Anything in the def is local to the function it defines, and separate from any other variables with the same name outside of that block.

A parameter can be made optional by defining a default value. Optional parameters must always come after mandatory parameters in Python.

def another_fun(confetti, fireworks, bassoons=None, candy='sugar fluff')

If you call another_fun with only two parameters, bassoons will default to None and candy will be set to the string 'sugar fluff'.

Answer 4

Why are we calling the print_line function and passing arguments to it that are named exactly the same as it's parameters?

That is really just a coincidence. The following does exactly the same as your example:

from sys import argv

script, input_encoding, error = argv

def main(language_file, what_encoding_do_you_want, list_of_errors):

    next_line = language_file.readline()

    if next_line:

        print_line(next_line, what_encoding_do_you_want, list_of_errors)
        return main(language_file, what_encoding_do_you_want, list_of_errors)

def print_line(line, encoding, errors):
    next_lang = line.strip()
    raw_bytes = next_lang.encode(encoding, errors=errors)
    cooked_string = raw_bytes.decode(encoding, errors=errors)

    print(raw_bytes, "<===>", cooked_string)

languages = open("languages.txt", encoding="utf-8")

main(languages, input_encoding, error)

It all comes down to 'scope'. The function-definition of print_line declares that it has three (positional) arguments, which can be referred to inside the function as 'line', 'encoding' and 'errors'.

The function-call to print_line from main adds three arguments to the call that refer to existing variables or arguments at that point in code: line refers to the variable created there; encoding and encoding refer to the argument to main function itself.