CODEWITHSUNDEEP
python-3.xcircleci
Jwan622
Jwan622

Reputation: 11639

Cannot install ElasticSearch on circleci due to permission errors

I have this error on elasticsearch:

Collecting aws-requests-auth
  Downloading https://files.pythonhosted.org/packages/a7/ba/e1601d0508b4150f8fe503f681079a7c9a17f7aa44e0d5cc42b9e3abdb8e/aws-requests-auth-0.4.2.tar.gz
Collecting boto3
  Downloading https://files.pythonhosted.org/packages/51/c4/d777bd125e6ef304ef52aa7c442a39b33b491922c6495ebd6663322508e4/boto3-1.10.10-py2.py3-none-any.whl (128kB)

     |████████████████████████████████| 133kB 44.0MB/s 
Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.6/site-packages (from elasticsearch==6.3.0->-r requirements.txt (line 1)) (1.25.6)
Requirement already satisfied: requests>=0.14.0 in /usr/local/lib/python3.6/site-packages (from aws-requests-auth->-r requirements.txt (line 2)) (2.22.0)
Collecting s3transfer<0.3.0,>=0.2.0
  Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB)

     |████████████████████████████████| 71kB 20.4MB/s 
Collecting jmespath<1.0.0,>=0.7.1
  Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting botocore<1.14.0,>=1.13.10
  Downloading https://files.pythonhosted.org/packages/28/69/78a1ee8d8c302c4cee6b088f2d166686b78135a86a65c5d2207b2964e438/botocore-1.13.10-py2.py3-none-any.whl (5.3MB)

     |████████████████████████████████| 5.4MB 58.9MB/s 
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/site-packages (from requests>=0.14.0->aws-requests-auth->-r requirements.txt (line 2)) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/site-packages (from requests>=0.14.0->aws-requests-auth->-r requirements.txt (line 2)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/site-packages (from requests>=0.14.0->aws-requests-auth->-r requirements.txt (line 2)) (2019.9.11)
Collecting python-dateutil<2.8.1,>=2.1; python_version >= "2.7"
  Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)

     |████████████████████████████████| 235kB 84.0MB/s 
Collecting docutils<0.16,>=0.10
  Downloading https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl (547kB)

     |████████████████████████████████| 552kB 62.4MB/s 
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/site-packages (from python-dateutil<2.8.1,>=2.1; python_version >= "2.7"->botocore<1.14.0,>=1.13.10->boto3->-r requirements.txt (line 3)) (1.12.0)
Building wheels for collected packages: aws-requests-auth
  Building wheel for aws-requests-auth (setup.py) ... - done
  Created wheel for aws-requests-auth: filename=aws_requests_auth-0.4.2-cp36-none-any.whl size=6831 sha256=12a613fe4a630e82d7909be436f3a5ae758b58a8475a4d4fadd24dcca3a12bce
  Stored in directory: /home/circleci/.cache/pip/wheels/3b/d7/e6/3bb2668c259b234ec78b7353f9063104dc449c1c89fc8e27f8
Successfully built aws-requests-auth
Installing collected packages: elasticsearch, aws-requests-auth, jmespath, python-dateutil, docutils, botocore, s3transfer, boto3
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/site-packages/elasticsearch-6.3.0.dist-info'
Consider using the `--user` option or check the permissions.

But I'm not sure why I'm getting the error. I am trying to use a virtualenv to install... shouldn't this get around the permission error?

version: 2
defaults: &defaults
  working_directory: ~/app/
  docker:
    - image: circleci/python:3.6
jobs:
  build_dataloader:
    <<: *defaults
    steps:
      - checkout
      - run:
          name: Setup virtualenv
          command: |
            virtualenv env
            source env/bin/activate
      - run:
          name: Install requirements
          command: |
            cd dataloader
            pip install -r requirements.txt
  dataloader_tests:
    <<: *defaults
    parallelism: 2
    steps:
      - checkout
      - run:
          name: Running dataloader tests
          command: |
            cd ~/app/dataloader
            python3 -m unittest discover tests/unit/
      - store_artifacts:
          path: test-reports/
          destination: app_tests
  build_extract:
    <<: *defaults
    working_directory: ~/app/extract
    steps:
      - checkout
      - run: ls -a
      - run:
          name: Setup virtualenv
          command: |
            virtualenv env
            source env/bin/activate
      - run:
          name: Install requirements
          command: |
            pip install -r requirements.txt
  extract_tests:
    <<: *defaults
    parallelism: 2
    steps:
      - checkout
      - run:
          name: Running extract tests
          command: |
            cd extract
            python3 -m unittest discover tests/unit/
      - store_artifacts:
          path: test-reports/
          destination: app_tests

workflows:
  version: 2
  run_tests:
    jobs:
      - build_dataloader
      - dataloader_tests:
          requires:
            - build_dataloader
      - build_extract
      - extract_tests:
          requires:
            - build_extract

Upvotes: 0

Views: 171

Answers (1)

Dylan Anthony
Dylan Anthony

Reputation: 682

Guessing a little bit here, but the error indicates that it's not installing to your virtualenv, it's installing to your base interpreter. This tells me that your source env/bin/activate is not applying to other steps (e.g. the run which contains pip install). So your options would be to include everything in the same run step as the source or put the source command in every run step.

Upvotes: 1

Related Questions