CODEWITHSUNDEEP
pythonpython-3.xexception
CaffeinatedMike
CaffeinatedMike

Reputation: 1607

In Python, will raising an Exception inside of a while loop lead to memory leaks?

I have the following code snippet that got me thinking 

# Utlizing api_sess from the /token response  
# api_sess is a requests.Session() with Authorization header set with session token

products_resp = api_sess.get(
    f'{API_BASE}/products',
    params = {'type': 'Item'}
)

if products_resp.ok:
    while True:
        current_page = products_resp.headers.get('X-Current-Page', 1)
        total_pages = products_resp.headers.get('X-Total-Pages', 0)
        products_json = products_resp.json()
        for product in products_json:
            print(product['groupCode'], product['sku'], product['productStatus'])
        # Handle pagination
        if current_page < total_pages:
            products_resp = api_sess.get(
                f'{API_BASE}/products',
                params = {'type': 'Item', 'page': current_page + 1}
            )
            if not products_resp.ok:
                # Break out of while loop before raising error
                break
        else:
            break
    # Raise exception outside of while loop to prevent memory leak
    if not products_resp.ok:
        raise ValueException(f'Request Error {products_resp.status_code}')
else:
    raise ValueException(f'Request Error {products_resp.status_code}')

Originally I was going to raise the exception inside of the while where I check the response of the next page request like so: 

if current_page < total_pages:
    products_resp = api_sess.get(
        f'{API_BASE}/products',
        params = {'type': 'Item', 'page': current_page + 1}
    )
    if not products_resp.ok:
        raise ValueException(f'Request Error {products_resp.status_code}')
else:
    break

At first I didn't think anything of it, but I started to wonder if resource cleanpup would take place since the error is raised inside an infinite while loop due to the True condition. Thus, why I inevitably reworked my logic to break out of the while loop first, then recheck response status.

In my opinion, this cautious method isn't as elegant as I'd like seeing as I'm duplicating the checking of the response status. Which leads me to my question:

Is it safe to keep the Exception raising inside the while loop or will it lead to memory leaks?

I know the question and scenario might seem trivial, but I'd like to stick with best practices in all aspects whenever possible. And upon my googling (while I didn't spend hours atleast) of the matter did not turn out any applicable explanations of how this scenario is handled.

Upvotes: 1

Views: 1122

Answers (1)

chepner
chepner

Reputation: 531768

Whether you leave the current scope via return or an uncaught exception, the reference count on the object will be decreased. A simple example:

import sys

a = "hello"

def foo(b):
    x = a
    print(f'x: {sys.getrefcount(x)}')
    if b:
        return
    else:
        raise Exception()

print(f'a: {sys.getrefcount(a)}')
foo(True)
print(f'a: {sys.getrefcount(a)}')

try:
    print(f'a: {sys.getrefcount(a)}')
    foo(False)
except Exception:
    pass
print(f'a: {sys.getrefcount(a)}')

The exact output may vary, but you should observe the reference count for x being one greater than the reference count for a, which remains constant before and after the calls to foo.

In both cases, the reference count for the global a increases when foo is called, and is decreased after foo exits (or after the try statement which catches the exception completes). If the exception is never caught, the interpreter exits anyway, making the question of a memory leak moot.

Upvotes: 3

Related Questions