CODEWITHSUNDEEP
pythondjangopostgresqldockeramazon-ec2
Joseph Hwang
Joseph Hwang

Reputation: 334

Why psycopg2 keep spitting out this error?

I'm very newbie at deploying stuff to somewhere. I'm working on deploying django rest api and react app to my ec2 instance. The instance is Ubuntu 20.04.2 LTS focal.

This is my project scheme

backend
\-backend(folder)
\-core(folder)
\-todos(folder)
\-uploads(folder)
\-users(folder)
\-db.sqlite3
\-Dockerfile
\-manage.py
\-Pipfile
\-Pipfie.lock
frontend
\-assets(folder)
\-public(folder)
\-src(folder)
\-Dockerfile
\-gulpfile.js
\-package-lock.json
\-package.json
\-tailwind.config.json
webserver
\-nginx-proxy.conf
docker-compose.yml

I made an dockerfile for the django rest api, and this is what it looks like. backend/Dockerfile

FROM python:3.7-slim
ENV PYTHONUNBUFFERED 1

...
ARGs and ENVs
...

RUN mkdir /backend
WORKDIR /backend
RUN pip3 install pipenv
COPY . /backend/
RUN pipenv install --python 3.7
RUN python manage.py makemigrations
RUN python manage.py migrate

But when I'm trying to do sudo docker-compose up at my ec2 instance, it spits out the error at RUN pipenv install. It spits out the error while locking Pipfile.lock. This is what error looks like.

[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 253, in install
[InstallError]:       site_packages=state.site_packages
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 2063, in do_install
[InstallError]:       keep_outdated=keep_outdated
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1312, in do_init
[InstallError]:       pypi_mirror=pypi_mirror,
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 900, in do_install_dependencies
[InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting psycopg2==2.8.6
[pipenv.exceptions.InstallError]:   Using cached psycopg2-2.8.6.tar.gz (383 kB)
[pipenv.exceptions.InstallError]:     ERROR: Command errored out with exit status 1:
[pipenv.exceptions.InstallError]:      command: /root/.local/share/virtualenvs/backend-gPBFdWVG/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-g7f_pd9f/psycopg2_4bbf83d4e03c4e2898c4116b32da6dbc/setup.py'"'"'; __file__='"'"'/tmp/pip-install-g7f_pd9f/psycopg2_4bbf83d4e03c4e2898c4116b32da6dbc/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-ad5nycuj
[pipenv.exceptions.InstallError]:          cwd: /tmp/pip-install-g7f_pd9f/psycopg2_4bbf83d4e03c4e2898c4116b32da6dbc/
[pipenv.exceptions.InstallError]:     Complete output (23 lines):
[pipenv.exceptions.InstallError]:     running egg_info
[pipenv.exceptions.InstallError]:     creating /tmp/pip-pip-egg-info-ad5nycuj/psycopg2.egg-info
[pipenv.exceptions.InstallError]:     writing /tmp/pip-pip-egg-info-ad5nycuj/psycopg2.egg-info/PKG-INFO
[pipenv.exceptions.InstallError]:     writing dependency_links to /tmp/pip-pip-egg-info-ad5nycuj/psycopg2.egg-info/dependency_links.txt
[pipenv.exceptions.InstallError]:     writing top-level names to /tmp/pip-pip-egg-info-ad5nycuj/psycopg2.egg-info/top_level.txt
[pipenv.exceptions.InstallError]:     writing manifest file '/tmp/pip-pip-egg-info-ad5nycuj/psycopg2.egg-info/SOURCES.txt'
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:     Error: pg_config executable not found.
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:     pg_config is required to build psycopg2 from source.  Please add the directory
[pipenv.exceptions.InstallError]:     containing pg_config to the $PATH or specify the full executable path with the
[pipenv.exceptions.InstallError]:     option:
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:         python setup.py build_ext --pg-config /path/to/pg_config build ...
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:     or with the pg_config option in 'setup.cfg'.
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:     If you prefer to avoid building psycopg2 from source, please install the PyPI
[pipenv.exceptions.InstallError]:     'psycopg2-binary' package instead.
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:     For further information please check the 'doc/src/install.rst' file (also at
[pipenv.exceptions.InstallError]:     <https://www.psycopg.org/docs/install.html>).
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:     ----------------------------------------
[pipenv.exceptions.InstallError]: WARNING: Discarding https://files.pythonhosted.org/packages/fd/ae/98cb7a0cbb1d748ee547b058b14604bd0e9bf285a8e0cc5d148f8a8a952e/psycopg2-2.8.6.tar.gz#sha256=fb23f6c71107c37fd667cb4ea363ddeb936b348bbd6449278eb92c189699f543 (from https://pypi.org/simple/psycopg2/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement psycopg2==2.8.6
[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for psycopg2==2.8.6
ERROR: Couldn't install package: psycopg2
 Package installation failed...
The command '/bin/sh -c pipenv install --python 3.7' returned a non-zero code: 1
ERROR: Service 'backend' failed to build

I tried

1.Installing stuff like python3-dev,libpq-dev,postgresql-client-12,postgresql-server-dev-12,postgresql-12,postgres. Also like postgresql-dev,postgresql-devel, which both not worked for me as it's saying unable to locate them. And I tried uninstalling psycopg2 to replace it to psycopy2-binary which still not worked.

2.Changing python:3.8.7-alpine3.13 to python:3.7-slim in Dockerfile above

3.Adding the directory containing pg_config to PATH(I found pg_config path by using sudo find / -name "pg_config" -print command)

# All the rest are same as default

PATH="/var/lib/docker/overlay2/083128b129fd82e7d4d2ffbae50eb833a8211ade5421110bf4ca2719c016d74b/diff/usr/bin:$PATH"
export PATH="/usr/lib/postgresql/12/bin:/usr/bin:$PATH"

None of the above worked for me. What should I do to fix it?

My backend/Pipfile

url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
djangorestframework = "*"
markdown = "*"
django-filter = "*"
pillow = "*"
django-cors-headers = "*"
djangorestframework-simplejwt = "*"
django = "==3.1.5"
django-dotenv = "*"
gunicorn = "*"
psycopg2 = "*"

[dev-packages]

[requires]
python_version = "3.7"

backend/Pipfile.lock

{
    "_meta": {
        "hash": {
            "sha256": "6d6662608d715726e4a18a5592c0ada7cdf57752d0de993e0829731c0ad62940"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "asgiref": {
            "hashes": [
                "sha256:5ee950735509d04eb673bd7f7120f8fa1c9e2df495394992c73234d526907e17",
                "sha256:7162a3cb30ab0609f1a4c95938fd73e8604f63bdba516a7f7d64b83ff09478f0"
            ],
            "markers": "python_version >= '3.5'",
            "version": "==3.3.1"
        },
        "django": {
            "hashes": [
                "sha256:2d78425ba74c7a1a74b196058b261b9733a8570782f4e2828974777ccca7edf7",
                "sha256:efa2ab96b33b20c2182db93147a0c3cd7769d418926f9e9f140a60dca7c64ca9"
            ],
            "index": "pypi",
            "version": "==3.1.5"
        },
        "django-cors-headers": {
            "hashes": [
                "sha256:1ac2b1213de75a251e2ba04448da15f99bcfcbe164288ae6b5ff929dc49b372f",
                "sha256:96069c4aaacace786a34ee7894ff680780ec2644e4268b31181044410fecd12e"
            ],
            "index": "pypi",
            "version": "==3.7.0"
        },
        "django-dotenv": {
            "hashes": [
                "sha256:3812bb0f4876cf31f902aad140f0645e120e51ee30eb7c40c22050f58a0e4adb",
                "sha256:a9b1b40a70bd321acd231926acedb9bd2c5e873e33a1873b34a7276d196a765e"
            ],
            "index": "pypi",
            "version": "==1.4.2"
        },
        "django-filter": {
            "hashes": [
                "sha256:84e9d5bb93f237e451db814ed422a3a625751cbc9968b484ecc74964a8696b06",
                "sha256:e00d32cebdb3d54273c48f4f878f898dced8d5dfaad009438fe61ebdf535ace1"
            ],
            "index": "pypi",
            "version": "==2.4.0"
        },
        "djangorestframework": {
            "hashes": [
                "sha256:0209bafcb7b5010fdfec784034f059d512256424de2a0f084cb82b096d6dd6a7",
                "sha256:0898182b4737a7b584a2c73735d89816343369f259fea932d90dc78e35d8ac33"
            ],
            "index": "pypi",
            "version": "==3.12.2"
        },
        "djangorestframework-simplejwt": {
            "hashes": [
                "sha256:7adc913ba0d2ed7f46e0b9bf6e86f9bd9248f1c4201722b732b8213e0ea66f9f",
                "sha256:bd587700b6ab34a6c6b12d426cce4fa580d57ef1952ad4ba3b79707784619ed3"
            ],
            "index": "pypi",
            "version": "==4.6.0"
        },
        "gunicorn": {
            "hashes": [
                "sha256:1904bb2b8a43658807108d59c3f3d56c2b6121a701161de0ddf9ad140073c626",
                "sha256:cd4a810dd51bf497552cf3f863b575dabd73d6ad6a91075b65936b151cbf4f9c"
            ],
            "index": "pypi",
            "version": "==20.0.4"
        },
        "markdown": {
            "hashes": [
                "sha256:5d9f2b5ca24bc4c7a390d22323ca4bad200368612b5aaa7796babf971d2b2f18",
                "sha256:c109c15b7dc20a9ac454c9e6025927d44460b85bd039da028d85e2b6d0bcc328"
            ],
            "index": "pypi",
            "version": "==3.3.3"
        },
        "pillow": {
            "hashes": [
                "sha256:165c88bc9d8dba670110c689e3cc5c71dbe4bfb984ffa7cbebf1fac9554071d6",
                "sha256:1d208e670abfeb41b6143537a681299ef86e92d2a3dac299d3cd6830d5c7bded",
                "sha256:22d070ca2e60c99929ef274cfced04294d2368193e935c5d6febfd8b601bf865",
                "sha256:2353834b2c49b95e1313fb34edf18fca4d57446675d05298bb694bca4b194174",
                "sha256:39725acf2d2e9c17356e6835dccebe7a697db55f25a09207e38b835d5e1bc032",
                "sha256:3de6b2ee4f78c6b3d89d184ade5d8fa68af0848f9b6b6da2b9ab7943ec46971a",
                "sha256:47c0d93ee9c8b181f353dbead6530b26980fe4f5485aa18be8f1fd3c3cbc685e",
                "sha256:5e2fe3bb2363b862671eba632537cd3a823847db4d98be95690b7e382f3d6378",
                "sha256:604815c55fd92e735f9738f65dabf4edc3e79f88541c221d292faec1904a4b17",
                "sha256:6c5275bd82711cd3dcd0af8ce0bb99113ae8911fc2952805f1d012de7d600a4c",
                "sha256:731ca5aabe9085160cf68b2dbef95fc1991015bc0a3a6ea46a371ab88f3d0913",
                "sha256:7612520e5e1a371d77e1d1ca3a3ee6227eef00d0a9cddb4ef7ecb0b7396eddf7",
                "sha256:7916cbc94f1c6b1301ac04510d0881b9e9feb20ae34094d3615a8a7c3db0dcc0",
                "sha256:81c3fa9a75d9f1afafdb916d5995633f319db09bd773cb56b8e39f1e98d90820",
                "sha256:887668e792b7edbfb1d3c9d8b5d8c859269a0f0eba4dda562adb95500f60dbba",
                "sha256:93a473b53cc6e0b3ce6bf51b1b95b7b1e7e6084be3a07e40f79b42e83503fbf2",
                "sha256:96d4dc103d1a0fa6d47c6c55a47de5f5dafd5ef0114fa10c85a1fd8e0216284b",
                "sha256:a3d3e086474ef12ef13d42e5f9b7bbf09d39cf6bd4940f982263d6954b13f6a9",
                "sha256:b02a0b9f332086657852b1f7cb380f6a42403a6d9c42a4c34a561aa4530d5234",
                "sha256:b09e10ec453de97f9a23a5aa5e30b334195e8d2ddd1ce76cc32e52ba63c8b31d",
                "sha256:b6f00ad5ebe846cc91763b1d0c6d30a8042e02b2316e27b05de04fa6ec831ec5",
                "sha256:bba80df38cfc17f490ec651c73bb37cd896bc2400cfba27d078c2135223c1206",
                "sha256:c3d911614b008e8a576b8e5303e3db29224b455d3d66d1b2848ba6ca83f9ece9",
                "sha256:ca20739e303254287138234485579b28cb0d524401f83d5129b5ff9d606cb0a8",
                "sha256:cb192176b477d49b0a327b2a5a4979552b7a58cd42037034316b8018ac3ebb59",
                "sha256:cdbbe7dff4a677fb555a54f9bc0450f2a21a93c5ba2b44e09e54fcb72d2bd13d",
                "sha256:cf6e33d92b1526190a1de904df21663c46a456758c0424e4f947ae9aa6088bf7",
                "sha256:d355502dce85ade85a2511b40b4c61a128902f246504f7de29bbeec1ae27933a",
                "sha256:d673c4990acd016229a5c1c4ee8a9e6d8f481b27ade5fc3d95938697fa443ce0",
                "sha256:dc577f4cfdda354db3ae37a572428a90ffdbe4e51eda7849bf442fb803f09c9b",
                "sha256:dd9eef866c70d2cbbea1ae58134eaffda0d4bfea403025f4db6859724b18ab3d",
                "sha256:f50e7a98b0453f39000619d845be8b06e611e56ee6e8186f7f60c3b1e2f0feae"
            ],
            "index": "pypi",
            "version": "==8.1.0"
        },
        "psycopg2": {
            "hashes": [
                "sha256:00195b5f6832dbf2876b8bf77f12bdce648224c89c880719c745b90515233301",
                "sha256:068115e13c70dc5982dfc00c5d70437fe37c014c808acce119b5448361c03725",
                "sha256:26e7fd115a6db75267b325de0fba089b911a4a12ebd3d0b5e7acb7028bc46821",
                "sha256:2c93d4d16933fea5bbacbe1aaf8fa8c1348740b2e50b3735d1b0bf8154cbf0f3",
                "sha256:56007a226b8e95aa980ada7abdea6b40b75ce62a433bd27cec7a8178d57f4051",
                "sha256:56fee7f818d032f802b8eed81ef0c1232b8b42390df189cab9cfa87573fe52c5",
                "sha256:6a3d9efb6f36f1fe6aa8dbb5af55e067db802502c55a9defa47c5a1dad41df84",
                "sha256:a49833abfdede8985ba3f3ec641f771cca215479f41523e99dace96d5b8cce2a",
                "sha256:ad2fe8a37be669082e61fb001c185ffb58867fdbb3e7a6b0b0d2ffe232353a3e",
                "sha256:b8cae8b2f022efa1f011cc753adb9cbadfa5a184431d09b273fb49b4167561ad",
                "sha256:d160744652e81c80627a909a0e808f3c6653a40af435744de037e3172cf277f5",
                "sha256:d5062ae50b222da28253059880a871dc87e099c25cb68acf613d9d227413d6f7",
                "sha256:f22ea9b67aea4f4a1718300908a2fb62b3e4276cf00bd829a97ab5894af42ea3",
                "sha256:f974c96fca34ae9e4f49839ba6b78addf0346777b46c4da27a7bf54f48d3057d",
                "sha256:fb23f6c71107c37fd667cb4ea363ddeb936b348bbd6449278eb92c189699f543"
            ],
            "index": "pypi",
            "version": "==2.8.6"
        },
        "pyjwt": {
            "hashes": [
                "sha256:a5c70a06e1f33d81ef25eecd50d50bd30e34de1ca8b2b9fa3fe0daaabcf69bf7",
                "sha256:b70b15f89dc69b993d8a8d32c299032d5355c82f9b5b7e851d1a6d706dffe847"
            ],
            "markers": "python_version >= '3.6'",
            "version": "==2.0.1"
        },
        "pytz": {
            "hashes": [
                "sha256:83a4a90894bf38e243cf052c8b58f381bfe9a7a483f6a9cab140bc7f702ac4da",
                "sha256:eb10ce3e7736052ed3623d49975ce333bcd712c7bb19a58b9e2089d4057d0798"
            ],
            "version": "==2021.1"
        },
        "sqlparse": {
            "hashes": [
                "sha256:017cde379adbd6a1f15a61873f43e8274179378e95ef3fede90b5aa64d304ed0",
                "sha256:0f91fd2e829c44362cbcfab3e9ae12e22badaa8a29ad5ff599f9ec109f0454e8"
            ],
            "markers": "python_version >= '3.5'",
            "version": "==0.4.1"
        }
    },
    "develop": {}
}

docker-compose.yml

version: '3'

services:
  backend:
    build:
      context: ./backend
      args:
        DJANGO_ALLOWED_HOSTS: stuff
        SECRET_KEY: stuff
        DJANGO_CORS_ORIGIN_WHITELIST: stuff
        BACKEND_ADMIN: stuff
        RDS_HOSTNAME: stuff
        RDS_PORT: stuff
        RDS_DB_NAME: stuff
        RDS_USERNAME: stuff
        RDS_PASSWORD: stuff
    command: gunicorn backend.wsgi --bind 0.0.0.0:8000
    ports:
      - "8000:8000"
  frontend:
    build:
      context: ./frontend
      args:
        BACKEND_API: stuff
    volumes:
      - build_folder:/frontend/build
  nginx:
    image: nginx:latest
    ports:
      - 80:8080
    volumes:
      - ./webserver/nginx-proxy.conf:/etc/nginx/conf.d/default.conf:ro
      - build_folder:/var/www/frontend
    depends_on:
      - backend
      - frontend
volumes:
  build_folder:

Upvotes: 1

Views: 6485

Answers (1)

Joseph Hwang
Joseph Hwang

Reputation: 334

For those who are struggling with the same issue above, I solved this problem by doing the things below:

  1. Not using pipenv install, instead using pipenv-to-requirements module for do a pip3 install -r requirements.txt, and therefore not using pipenv

  2. Not using alpine image of python, instead using python:3.8.7 which is from the docker hub, and is their defacto image.

Hope this helps for someone who's struggling with the issue like the one above!!

Upvotes: 2

Related Questions