CODEWITHSUNDEEP
pythonmysqldjango
Bart P.
Bart P.

Reputation: 929

'MySQL server has gone away' with longlasting processes

We use Django to run modelsimulations. The code runs as a cronscript (not via wsgi) and registers whether the simulation ran sucessfully or not. At the beginning of a modelsimulation a statusobject is created with status RUNNING. At the end of the modelsimulation this statusobject is updated to SUCCESS or FAILED. The (pseudo)code looks like this:

def mainRoutine():
    myStatusObject = createStatusObject()
    try:
        runModelSimulation()  # this runs the modelsimulation
    except:
        updateStatus(myStatusObject,'FAILED')
    updateStatus(myStatusObject,'SUCCESS')

def createStatusObject():
    myStatusObject = models.StatusObject()
    myStatusObject.task = 'somename'
    myStatusObject.status = 'RUNNING'
    myStatusObject.save()
    return myStatusObject

def updateStatus(myStatusObject, newstatus):
    myStatusObject.status = newstatus
    myStatusObject.save()

The problem starts when the modelsimulation takes a long time (simulations of several hours to days are possible). Then there is a connection error to the MySQL-database, with error 'MySQL server has gone away'. I understand from other posts that this is linked to some connection remaining with the database (which can be solved by fiddling with the serverside configuration of MySQL).

For the moment I found a workaround, by passing the ID of the statusobject, rather than the statusobject instance. In the updateStatus-subroutine I then use a get(ID=thisID) to retrieve the correct statusObject instance before updating. This does not yield a time-out.

def updateStatus(myStatusObjectID, newstatus):
    myStatusObject = StatusObject.objects.get(id=myStatusObjectID)
    myStatusObject.status = newstatus
    myStatusObject.save()

Problem solved?! However, we have other objects similar to statusobject and are likely to encounter analogue problems. So I would like to understand why this connection remains open. What is the difference between passing the StatusObject instance, and passing the StatusObject ID combined with .get()? At what point is the connection made, and how could I prevent this connection from remaining open? Can we tell django to close the connection after .save(), and reopen it on the next update of the instance. Also because it is easier to pass around a StatusObject instance, than to search for it based and reopen it based on its attribute.

Upvotes: 0

Views: 139

Answers (1)

catavaran
catavaran

Reputation: 45575

Close the connection to MySQL before the modelsimulation. Django will automatically reconnect to sql server on myStatusObject.save() call.

from django.db import connection

def mainRoutine():
    myStatusObject = createStatusObject()
    connection.close()
    try:
        runModelSimulation()  # this runs the modelsimulation
    except:
        updateStatus(myStatusObject,'FAILED')
    updateStatus(myStatusObject,'SUCCESS')

Upvotes: 2

Related Questions