CODEWITHSUNDEEP
pythonsqldjango
xle
xle

Reputation: 310

Django OperationalError: Parser Stack Overflow

I have a multi-page form for an accounting app in my project. The first page allows the user to select multiple creditors, the second to select multiple debtors, and the third generates a table so they can input the credit from each debtor to each creditor.

Each page of the form passes the selected data to the next page via POST (using the UUID of the accounts in question), and the final page has the "submit" button that actually generates the transaction object in the database.

When I use the form with a large number of debtors or creditors selected, I get an "OperationalError: Parser Stack Overflow."

From the only other question on this topic I can find (django.db.utils.OperationalError: parser stack overflow) it looks like I need to either find a slimmer way of transmitting that data (perhaps by creating an "intermediary" model in the database to hold the data from each page so that each page need only be passed a single object). However, as you can see from the error below, it is the "account_list" that causes the overflow. This list comes from the query:

Account.objects.all().order_by('type')

So I am unsure that this would solve the problem.

Alternatively, I could increase the size of the SQL parser stack size. However, I am unable to recompile sqlite on my host system as it uses virtual hosting and I am not a sysadmin.

Any suggestions?

The "local vars" data from the django debug page is below:

__class__   

<class 'Bank.views.CreateTransactionData'>

account     

<Account: (User) User Z>

account_list    

Error in formatting: OperationalError: parser stack overflow

all_accounts    

<QuerySet [<Account: (User) User A>, <Account: (User) User B>, <Account: (User) User C>, <Account: (User) User D>, <Account: (User) User E>, <Account: (User) User F>, <Account: (User) User G>, <Account: (User) User H>, <Account: (User) User I>, <Account: (User) User J>, <Account: (User) User K>, <Account: (User) User L>, <Account: (User) User M>, <Account: (User) User N>, <Account: (User) User O>, <Account: (User) User P>, <Account: (User) User Q>, <Account: (User) User R>, <Account: (User) User S>, <Account: (User) User T>, '...(remaining elements truncated)...']>

args    

()

creditor    

<Account: (User) User A>

data    

{'0497dfe8-37c5-4939-8dd9-b9c8093d0ae1': 'TRUE',
 '07dbb8d3-cc1f-497f-ab3b-4a5d815cac3a': 'TRUE',
 '0957bfcd-7c16-4559-8e23-57a057c36155': 'TRUE',
 '26fed283-450b-4fe3-8ac6-84dadd67e950': 'TRUE',
 '3129ffe8-0ece-4576-afed-00eeb789ad80': 'TRUE',
 '517ca471-bf9a-40c6-9988-b773da21d30d': 'TRUE',
 '51f4a86f-b5d8-4910-b338-b461c544016d': 'TRUE',
 '5706826c-c895-4063-823f-2d98fdb012ff': 'TRUE',
 '57baa54a-670a-4310-baf3-5bd0023f96a4': 'TRUE',
 '61b91670-5424-4625-a964-8dd257524ee4': 'TRUE',
 '6ae151ff-4e5d-4972-b6a5-16efd819c53b': 'TRUE',
 '6cfecce8-95b2-468c-80d6-b52af2ed2037': 'TRUE',
 '714c8d14-bc4d-47ad-9fae-88366064ade8': 'TRUE',
 '79168ba9-f187-404f-968a-c9224d570d3b': 'TRUE',
 '9d47c7dc-cf6c-4090-a40b-f7c85eaccc37': 'TRUE',
 '9fd1e279-1b70-4ef1-8b79-98ba3d668542': 'TRUE',
 'a1a11ebe-b5d5-49bf-a60f-4f33dafdc5a7': 'TRUE',
 'a5909c0b-d991-4b0e-bb58-ccef7dbdf32f': 'TRUE',
 'a8d74160-50bd-48ce-9862-ba48be6a4619': 'TRUE',
 'b66ed030-d40b-413e-83ec-47082d3281c9': 'TRUE',
 'ca50f233-8118-4e1f-a951-c6a38bd3f285': 'TRUE',
 'creditor': 'a8d74160-50bd-48ce-9862-ba48be6a4619',
 'csrfmiddlewaretoken': 'DwbBCOSdTpV2HhZYgCodQBpF3Ut167EAUfVc8f8AOA04PKTVI4vyn1crFwpSfTGd',
 'd3040f2c-edd1-4466-878e-23aaed0e35e3': 'TRUE',
 'd660841d-1268-4a6a-8b30-5d9cc8157989': 'TRUE',
 'date': '2019-08-29',
 'e4032076-98a7-4a4b-bfc4-d7d599056a77': 'TRUE',
 'ee6880aa-dc23-4827-8598-c6a67766af61': 'TRUE',
 'f0182a4c-c221-45d1-95ac-dd2012111402': 'TRUE'}

date    

datetime.datetime(2019, 8, 29, 0, 0)

key     

UUID('714c8d14-bc4d-47ad-9fae-88366064ade8')

kwargs  

{}

request     

<WSGIRequest: POST '/Bank/Transaction/Create/Data/'>

self    

<Bank.views.CreateTransactionData object at 0x053836D0>

Upvotes: 0

Views: 307

Answers (1)

xle
xle

Reputation: 310

Sorted it!

For anyone else experiencing this problem, the issue was with the way I extracted the list of selected accounts.

I had:

        # extract list of debtors
        all_accounts = Account.objects.all()
        account_list = Account.objects.none()
        for account in all_accounts:
            key = account.account_key
            if str(key) in data.keys() and data.get(str(key)) == 'TRUE':
                account_list = account_list.union(all_accounts.filter(account_key=key))
        if not account_list.first(): # Check for no debtors selected
            raise Http404('You must have debtors in your transaction.') # Complain

Which created a new query for every account you selected...

Changed it to:

        # extract list of debtors
        all_accounts = Account.objects.all().order_by('type')
        account_list = []
        for account in all_accounts:
            key = account.account_key
            if str(key) in data.keys() and data.get(str(key)) == 'TRUE':
                account_list.append(account)
        if not account_list: # Check for no debtors selected
            raise Http404('You must have debtors in your transaction.') # Complain

Moral of the story: be careful with the .union() method... And maybe attend some CS lectures before you try to build an accounting system from scratch.

Upvotes: 1

Related Questions