CODEWITHSUNDEEP
python-3.xgoogle-cloud-platformgoogle-cloud-storage
mrtksy
mrtksy

Reputation: 113

Google Cloud Storage get_blob call results in stale socket connections

I am trying to get a blob object from a google storage bucket using python:

from google.cloud import storage
storage_bucket = storage.Client(project).bucket(bucket_name)
blob = storage_bucket.get_blob("some-blob-path")

Doing this results in stale socket connections that are first in ESTABLISHED status and then switch to CLOSE_WAIT status after a while:

python3.6 33713 xxxxx   14u    IPv6 0xfed2daa65e1d7abf        0t0        TCP [2601:646:9500:d6cf:2061:3a97:e840:70e4]:52658->sfo03s08-in-x0d.1e100.net:https (CLOSE_WAIT)
python3.6 33713 xxxxx   15u    IPv6 0xfed2daa65e1d807f        0t0        TCP [2601:646:9500:d6cf:2061:3a97:e840:70e4]:52659->nuq04s29-in-x0a.1e100.net:https (CLOSE_WAIT)
python3.6 33713 xxxxx   16u  KQUEUE                                          count=0, state=0xa
python3.6 33713 xxxxx   18u    IPv6 0xfed2daa66543b07f        0t0        TCP [2601:646:9500:d6cf:2061:3a97:e840:70e4]:52664->sfo03s08-in-x0d.1e100.net:https (CLOSE_WAIT)
python3.6 33713 xxxxx   19u    IPv6 0xfed2daa66543aabf        0t0        TCP [2601:646:9500:d6cf:2061:3a97:e840:70e4]:52665->nuq04s29-in-x0a.1e100.net:https (CLOSE_WAIT)
python3.6 33713 xxxxx   20u    IPv6 0xfed2daa65e1ccf3f        0t0        TCP [2601:646:9500:d6cf:2061:3a97:e840:70e4]:52670->sfo03s08-in-x0d.1e100.net:https (CLOSE_WAIT)
python3.6 33713 xxxxx   21u    IPv6 0xfed2daa65e1cacbf        0t0        TCP [2601:646:9500:d6cf:2061:3a97:e840:70e4]:52671->nuq04s29-in-x0a.1e100.net:https (CLOSE_WAIT)

To be precise, each call to get_blob results in two of those CLOSE_WAIT statuses. When this happens too many times in the running code, I end up getting "too many connections open" error, after which the only solution would be to restart my daemon.

Has anyone experienced this issue before? Is there a specific clean-up that I should do? (just deleting the object does not work)

Thanks

Upvotes: 3

Views: 315

Answers (1)

Mangu
Mangu

Reputation: 3325

A good solution for this, as the classes you are using don't have a proper close method or similar, is to raise a issue in the official Github repository, as they are the maintainers of this library, and they are better suited to fix this issue.

Upvotes: 1

Related Questions