Odoo - Large processes always turns to an InternalError at last

Question

we have odoo v8 installed on server and doing import process for large file, these process take at least 15 min, but most of the times after couple of minutes we will get this error.

Odoo Server Error

Traceback (most recent call last):
  File "/opt/odoo/odoo/openerp/http.py", line 537, in _handle_exception
    return super(JsonRequest, self)._handle_exception(exception)
  File "/opt/odoo/odoo/openerp/http.py", line 574, in dispatch
    result = self._call_function(**self.params)
  File "/opt/odoo/odoo/openerp/http.py", line 310, in _call_function
    return checked_call(self.db, *args, **kwargs)
  File "/opt/odoo/odoo/openerp/service/model.py", line 118, in wrapper
    return f(dbname, *args, **kwargs)
  File "/opt/odoo/odoo/openerp/http.py", line 307, in checked_call
    return self.endpoint(*a, **kw)
  File "/opt/odoo/odoo/openerp/http.py", line 803, in __call__
    return self.method(*args, **kw)
  File "/opt/odoo/odoo/openerp/http.py", line 403, in response_wrap
    response = f(*args, **kw)
  File "/opt/odoo/odoo/addons/web/controllers/main.py", line 948, in call_button
    action = self._call_kw(model, method, args, {})
  File "/opt/odoo/odoo/addons/web/controllers/main.py", line 936, in _call_kw
    return getattr(request.registry.get(model), method)(request.cr, request.uid, *args, **kwargs)
  File "/opt/odoo/odoo/openerp/api.py", line 250, in wrapper
    return old_api(self, *args, **kwargs)
  File "/opt/odoo/odoo/openerp/api.py", line 372, in old_api
    result = method(recs, *args, **kwargs)
  File "/opt/odoo/emipro_custom_modules/reorder_calculation_ept/wizard/import_forecasted_sale.py", line 204, in do_import
    self.create_log(str(e),log_id)
  File "/opt/odoo/odoo/openerp/api.py", line 248, in wrapper
    return new_api(self, *args, **kwargs)
  File "/opt/odoo/emipro_custom_modules/reorder_calculation_ept/wizard/import_forecasted_sale.py", line 112, in create_log
    'log_type':log_type,
  File "/opt/odoo/odoo/openerp/api.py", line 248, in wrapper
    return new_api(self, *args, **kwargs)
  File "/opt/odoo/odoo/openerp/models.py", line 4092, in create
    record = self.browse(self._create(old_vals))
  File "/opt/odoo/odoo/openerp/api.py", line 248, in wrapper
    return new_api(self, *args, **kwargs)
  File "/opt/odoo/odoo/openerp/api.py", line 481, in new_api
    result = method(self._model, cr, uid, *args, **kwargs)
  File "/opt/odoo/odoo/openerp/models.py", line 4230, in _create
    tuple([u[2] for u in updates if len(u) > 2])
  File "/opt/odoo/odoo/openerp/sql_db.py", line 158, in wrapper
    return f(self, *args, **kwargs)
  File "/opt/odoo/odoo/openerp/sql_db.py", line 234, in execute
    res = self._obj.execute(query, params)
InternalError: current transaction is aborted, commands ignored until end of transaction block

Multi worker has been configured on server so that might be an issue (not sure). And I didn't fire any query for that process so that's not related to query failed issue (I read at many questions that issue is related not to roll backed failed query), but in that case there is no query, I simply read xls file to dictionary and then process that dictionary.

Config file

db_maxconn = 500
max_cron_threads=20
osv_memory_age_limit = 1.0
osv_memory_count_limit = False
data_dir = /opt/odoo/file_store
logfile = /opt/odoo/log/odoo.log
logrotate = True
workers = 17
limit_time_cpu = 15300
;limit_time_cpu = 6000
;limit_time_real = 3000
limit_time_real = 17500
;limit_memory_soft = 1810720000
;limit_memory_hard = 2177721600
bin_path=/usr/local/bin

These will happen with the many other processes which will take more time to do the operations. Can any one please guide us ?

Answer 1

With workers enabled you should try to play with the config parameters limit-time-cpu and limit-time-real. These are set to something like 60 sec by default, so processes will be killed after this time, without any warning. Maybe the config settings limit-memory-* are interesting here, too.