CODEWITHSUNDEEP
pythonsockets
Harvey
Harvey

Reputation: 1418

Python 3 Sockets - Can I keep a socket open while stopping and re-running a program?

I've been scratching my head trying to figure out if this is possible.

I have a server program running with about 30 different socket connections to it from all over the country. I need to update this server program now and although the client devices will automatically reconnect, its not totally reliable.

I was wondering if there is a way of saving the socket object to a file? then load it back up when the server restarts? or forcefully keeping a socket open even after the program stops. This way the clients never disconnect at all.

Could really do with hot swappable code here really!

Upvotes: 2

Views: 712

Answers (3)

freakish
freakish

Reputation: 56557

Solution 1.

It can be done with some process magic, at least under linux (although I do believe similar windows api exists). First of all note that sockets cannot be stored in a file. These objects are temporary by their nature. But you can keep them in a separate process. Have a look at this:

Can I open a socket and pass it to another process in Linux

So one way to accomplish this is the following:

  1. Create a "keeper" process at some point (make sure that the process is not a child of the main process so that it stays alive when the main process is gone)
  2. Send all sockets to the keeper process via sendmsg() with SCM_RIGHTS
  3. Shutdown the main process
  4. Do whatever update you have to
  5. Fire the main process
  6. Retrieve sockets from the keeper process
  7. Shutdown the keeper process

However this solution is quite difficult to maintain. You have two separate processes, it is unclear which is the master and which is a slave. So you would probably need another master process at the top. Things get nasty very quickly, not to mention security issues.

Solution 2.

Reloading modules as suggested by @gavinb might be a solution. Note however that in practice this often breaks the app. You never know what those modules do under the hood unless you know the code of every single Python file you use. Plus it imposes some restrictions on modules, i.e. they have to be reloadable. For example some modules use inline caching which makes reloading difficult.

Also once a module is loaded in a different module it keeps a reference to that module. So you not only have to reload it but also update references in every other module that loaded it earlier. The maintanance costs raise very quickly unless you thought about it at the begining of the project (so that every import is encapsulated for easy reload). And bugs caused by two different versions of a module running in the same process are (I imagine, never been in this situation though) extremely difficult to find.

Anyway I would avoid that.

Solution 3.

So this is XY problem. Instead of saving sockets how about you put a proxy in front of the main server? IMO this is the safest and at the same time simpliest solution. The proxy will communicate with the main server (for example over unix domain sockets) and will buffer the data and automatically reconnect to the main server once it is available again. Perhaps you can even reuse some existing tech, e.g. nginx.

Upvotes: 3

Sraw
Sraw

Reputation: 20234

I think your critical question is how to hot reload.

And as mentioned by @gavinb, you can import importlib and then use importlib.reload(module) to reload a module dynamically.

Be careful, the parameter of reload(param) must be a module.

Upvotes: 0

gavinb
gavinb

Reputation: 20048

No, the sockets are special file handles that belong to the process. If you close the process, the runtime will force close any open files/sockets. This is not Python specific; it is just how operating systems manage resources.

Now what you can do however is dynamically reload one or more modules while keeping the process active. It might take some careful management when you have open sockets, but in theory it should be possible. So yes, hot swappable code is actually supported by Python.

Do some reading and research on "dynamic reloading". The importlib module in Python 3 provides the reload function which is used to:

Reload a previously imported module. The argument must be a module object, so it must have been successfully imported before. This is useful if you have edited the module source file using an external editor and want to try out the new version without leaving the Python interpreter.

Upvotes: 2

Related Questions