CODEWITHSUNDEEP
pythonredisredis-py
kellycup8
kellycup8

Reputation: 141

Does initializing many redis-py connections slow down server?

Let's say on my python server I open 10 different redis connections like this:

import redis

conn1 = redis.Redis(host=aaaa)
conn2 = redis.Redis(host=bbbb)
conn3 = redis.Redis(host=cccc)
...
conn10 = redis.Redis(host=jjjj)

How is redis-py operating under the hood, and will my server be slowed down with each additional redis.Redis() call?

Upvotes: 1

Views: 821

Answers (1)

Simba
Simba

Reputation: 27588

redis-py manages connections with ConnectionPool, and it doesn't pre-fill the connection pool. A connection is created only a command needs being sent.

Following source code could be enough to help understand the process. Read source code of redis-py if you wanna dig it deeper.

class Redis:
    def set(self, name, value,
            ex=None, px=None, nx=False, xx=False, keepttl=False, get=False):
            ...

            return self.execute_command('SET', *pieces, **options)

    def execute_command(self, *args, **options):
        ...
        conn = self.connection or pool.get_connection(command_name, **options)
        try:
            conn.send_command(*args)
            return self.parse_response(conn, command_name, **options)
            ...

ConnectionPool.get_connection() is where a conn created. In fact it's the underlying socket being created.

class ConnectionPool:
    def get_connection(self, command_name, *keys, **options):
            ...
            try:
            # ensure this connection is connected to Redis
            connection.connect()
            ...

class Connection:
    ...
    def _connect(self):
        "Create a TCP socket connection"
        # we want to mimic what socket.create_connection does to support
        # ipv4/ipv6, but we want to set options prior to calling
        # socket.connect()
        err = None
        for res in socket.getaddrinfo(self.host, self.port, self.socket_type,
                                      socket.SOCK_STREAM):
            family, socktype, proto, canonname, socket_address = res
            sock = None
            try:
                sock = socket.socket(family, socktype, proto)
                ...

Upvotes: 1

Related Questions