CODEWITHSUNDEEP
docker
Johiasburg Frowell
Johiasburg Frowell

Reputation: 588

Scrapoxy install error using docker

While trying to get started with Scrapoxy as per the instructions here, I had followed the instructions until step 3A. However, when I run docker using the command specified under step 3A, I get an error which keeps repeating. Below is the error, along with the outputs preceding it each time it shows up:

2017-11-08T22:24:51.804Z - debug: [Manager] checkInstances
2017-11-08T22:24:52.515Z - debug: [Manager] adjustInstances: required:1 / actual:0
2017-11-08T22:24:52.515Z - debug: [Manager] adjustInstances: add 1 instances
2017-11-08T22:24:52.515Z - debug: [ProviderAWSEC2] createInstances: count=1
2017-11-08T22:24:53.225Z - debug: [ProviderAWSEC2] createInstances: actualCount=0
2017-11-08T22:24:54.045Z - error: [Manager] Error: Cannot update or adjust instances: InvalidParameterValue: Value () for parameter groupId is invalid. The value cannot be empty
    at Request.extractError (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/services/ec2.js:50:35)
    at Request.callListeners (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:615:14)
    at Request.transition (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:617:12)
    at Request.callListeners (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:615:14)
    at Request.transition (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:38:9)

When I open the Scrapoxy GUI (as per step 4 in the Quick Start guide linked to above), no instances are shown, which I expected, based on the output shown above. Step 6 in the quickstart returns an error saying scrapoxy: command not found

How can I remedy this, so I can use Scrapoxy? I would like to integrate it with the python library Scrapy.

(I am using Ubuntu 16.04, if it matters. Docker is already installed and functional.)

Upvotes: 0

Views: 556

Answers (1)

user8389458
user8389458

Reputation:

What's wrong with your step 3A

  • Reference: Scrapoxy Issue NO.70
  • Make sure your AWS instance met following criterias:
    • It's located in eu-west1 region
    • You have create a security group in eu-west1 region

What's wrong with your step 3B

  • You didn't install scrapoxy properly.

  • Here is an example of how to install scrapoxy on Ubuntu 16.04 LTS (tested on Linode)

    • You should first install nodejs on your Ubuntu 16.04 via 

      sudo apt-get update
curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash -
sudo apt-get install -y nodejs

    • Then Install Make and Scrapoxy:

      sudo apt-get install -y build-essential && sudo npm install -g scrapoxy

    • Generate Configuration:

      scrapoxy init conf.json
    • Install your favorite Text Editor(Vim or etc..)
    • Edit configuration
      • Edit conf.json
      • In the commander section, replace password by a password of your choice
      • In the providers/awsec2 section, replace accessKeyId, secretAccessKey and region by your AWS credentials and parameters.

    • Start Scrapoxy

      scrapoxy start conf.json -d

    • Connect GUI Interface:

      http://server-ip:8889

Upvotes: 2

Related Questions