CODEWITHSUNDEEP
pythonpyramid
gosom
gosom

Reputation: 1319

Pyramid global object share across thread

I have a small pyramid web service.

I have also a python class that creates an index of items and methods to search fast across them. Something like:

class MyCorpus(object):

    def __init__(self):
        self.table = AwesomeDataStructure()

    def insert(self):
        self.table.push_back(1)

    def find(self, needle):
        return self.table.find(needle)

I would like to expose the above class to my api.

I can create only one instance of that class (memory limit).

So I need to be able to instantiate this class before the server starts. And my threads should be able to access it.

I also need some locking mechanism(conccurrent inserts are not supported).

What is the best way to achieve that?

Upvotes: 4

Views: 2046

Answers (2)

Ilja Everilä
Ilja Everilä

Reputation: 52929

Add an instance of your class to the global application registry during your Pyramid application's configuration:

config.registry.mycorpus = MyCorpus()

and later, for example in your view code, access it through a request:

request.registry.mycorpus

You could also register it as a utility with Zope Component Architecture using registry.registerUtility, but you'd need to define what interface MyCorpus provides etc., which is a good thing in the long run. Either way having a singleton instance as part of the registry makes testing your application easier; just create a configuration with a mock corpus.

Any locking should be handled by the instance itself:

from threading import Lock

class MyCorpus(object):

    def __init__(self, Lock=Lock):
        self.table = AwesomeDataStructure()
        self.lock = Lock()

    ...

    def insert(self):
        with self.lock:
            self.table.push_back(1)

Upvotes: 6

Sergey
Sergey

Reputation: 12417

Any global variable is shared between threads in Python, so this part is really easy: "... create only one instance of that class ... before the server starts ... threads should be able to access it":

corpus = MyCorpus()  # in global scope in any module

Done! Then import the instance from anywhere and call your class' methods:

from mydata import corpus

corpus.do_stuff()

No need for ZCA, plain pythonic Python :)

(the general approach of keeping something large and very database-like within the webserver process feels quite suspicious though, I hope you know what you're doing. I mean - persistence? locking? sharing data between multiple processes? Redis, MongoDB and 1001 other database products have those problems solved)

Upvotes: 2

Related Questions