CODEWITHSUNDEEP
haproxy
Mario
Mario

Reputation: 1488

Make rotating proxies both HTTP and HTTPS

I am using mattes's rotating proxy docker container from GitHub -> https://github.com/mattes/rotating-proxy

I've setup everything correctly and it is running perfectly.

However, it is acting as a HTTPS proxy only and not both HTTP and HTTPS.

I've changed the HAProxy config file and set the mode from http to tcp but it is still HTTPS only.

This is my current haproxy.cfg.erb file which is changed to haproxy.cfg later using a Ruby script:

global
  maxconn 1024000 
  daemon
  pidfile <%= pid_file %>

defaults
  mode tcp
  maxconn 1024000
  option  tcplog
  option  dontlognull
  retries 3
  timeout connect 5s
  timeout client 60s
  timeout server 60s


listen stats *:4444
  mode            http
  log             global
  maxconn 1024000
  clitimeout      100s
  srvtimeout      100s
  contimeout      100s
  timeout queue   100s
  stats enable
  stats hide-version
  stats refresh 30s
  stats show-node
  stats uri /haproxy?stats


frontend rotating_proxies
  bind *:<%= port %>
  mode tcp
  default_backend tor
  option http_proxy
  option tcplog

backend tor
  option http_proxy
  option tcplog
  mode tcp
  balance leastconn # http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#balance

  <% backends.each do |b| %>
  server <%= b[:name] %><%= b[:port] %> <%= b[:addr] %>:<%= b[:port] %>
  <% end %>

I am using http://www.checker.freeproxy.ru/checker/ and it says the proxy type is HTTPS only.

Any help would be appreciated.

Thanks!

Upvotes: 2

Views: 2619

Answers (2)

protld
protld

Reputation: 440

All the scripts you're trying to us (TOR rotating proxies) are simply using the TOR bridges to get you a new proxy on each request.

Well, you've to know that TOR is SOCKS only and this is clear on the README file of the GitHub repository you're trying to use.

Please note: Tor offers a SOCKS Proxy only. In order to allow communication from HAproxy to Tor, Polipo is used to translate from HTTP proxy to SOCKS proxy. HAproxy is able to talk to HTTP proxies only.

And the answer of @usamember is totally useless and you might notice that you've already breaked your HAProxy configuration by following his answer. Now you should receive "504: Connection refused"

However, there's an alternative to TOR. See this GitHub repository: https://github.com/jgontrum/proxies-rotator

It uses GimmeProxy.com's API to get proxies and rotate them on your client. So this is an alternative to TOR

Upvotes: 4

usamember
usamember

Reputation: 488

In your backend tor you have this option:

option http_proxy

You've to remove this line. Also this proxies checker that you're using will always say HTTPS because this is a rotating proxies client.

Upvotes: 0

Related Questions