Configuration for using Mercurial with Bitbucket from behind a certificate rewriting proxy?

Question

I'm trying to access BitBucket from work. The only access to the Internet is via an authenticating HTTP proxy which proxies http on port 8080 and SSL on port 8070. This proxy conducts a man-in-the-middle attack on SSL connections, browsers are able to create HTTPS connections to the Internet only due to the installation of a fake Websense certificate on all clients.

I am able to connect to BitBucket using Git but not using Mercurial. I'm using Mercurial version 2.0.2.

With Git I use the following config in .gitconfig

[user]
    name = Firstname Lastname
    email = firstname_lastname@domain.co.uk
[http]
    proxy = http://name:password@nnn.nnn.nnn.nnn:8080

And can clone a repository with the following command

D:\MercurialTesting>git clone http://Firstname_Lastname@bitbucket.org/Firstname_Lastname/bb102repo.git test1
Cloning into 'test1'...
Password for 'bitbucket.org':
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.

D:\MercurialTesting>

With the addition of this config setting

[http]
    sslverify = false

I can also clone the repository via the https url https://Firstname_Lastname@bitbucket.org/Firstname_Lastname/bb102repo.git

Using Mercurial though it's a different story. Using the following config in mercurial.ini

[http_proxy]
host = nnn.nnn.nnn.nnn:8080
user = firstname_lastname@domain.co.uk
passwd = password

Mercurial will access my own Mercurial server at home no problem.

D:\MercurialTesting>hg --debug clone http://nnn.nnn.nnn.nnn/hg/Workspaces/Test1
using http://nnn.nnn.nnn.nnn/hg/Workspaces/Test1
proxying through http://nnn.nnn.nnn.nnn:8080
sending capabilities command
http authorization required
realm: Mercurial Repositories
user: username
password:
http auth: user username, password *******
destination directory: Test1
query 1; heads
sending batch command
http auth: user username, password *******
requesting all changes
sending getbundle command
http auth: user username, password *******
adding changesets
changesets: 1 chunks
add changeset 711ff2c6f5b2
changesets: 2 chunks
add changeset 9034b963b4c1
. . .

Using the exact same configuration and trying to access BitBucket through Mercurial just hangs.

D:\MercurialTesting>hg --debug clone http://bitbucket.org/Firstname_Lastname/bb101repo
using http://bitbucket.org/Firstname_Lastname/bb101repo
proxying through http://nnn.nnn.nnn.nnn:8080
sending capabilities command
abort: error: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

D:\MercurialTesting>

Using the same configuration with SSL via the url https://bitbucket.org/Firstname_Lastname/bb101repo Mercurial hangs in exactly the same way. During this process Wireshark detects no network activity at all.

Changing the settings in Mercurial.ini to match the port which the proxy serves SSL through has no effect. Setting the environment variable http_proxy makes no difference, but setting the environment variable https_proxy changes the output completely. Setting https_proxy and adding --insecure to the hg command invocation results in:

D:\MercurialTesting>hg --debug clone http://bitbucket.org/Firstname_Lastname/bb101repo --insecure
using http://bitbucket.org/Firstname_Lastname/bb101repo
proxying through http://nnn.nnn.nnn.nnn:8080
sending capabilities command
warning: bitbucket.org certificate with fingerprint 79:ce:0d:30:b0:17:29:6a:d1:9f:dd:d3:62:80:70:28:5e:9f:c2:e3 not verified (check hostfingerprints or web.cacerts config setting)
http authorization required
realm: Bitbucket.org HTTP
user: Firstname_Lastname
password:
http auth: user Firstname_Lastname, password ***
warning: bitbucket.org certificate with fingerprint 79:ce:0d:30:b0:17:29:6a:d1:9f:dd:d3:62:80:70:28:5e:9f:c2:e3 not verified (check hostfingerprints or web.cacerts config setting)
abort: HTTP Error 502: Success

D:\MercurialTesting>

And now Wireshark does detect an exchange taking place between my workstation and the proxy server. What I find most confusing though is that it doesn't make the slightest difference what I set https_proxy to, hg always uses the http proxy setting from Mercurial.ini and produces the same output above irrespective of whether I set https_proxy to the correct details for the SSL proxy or to complete garbage. The only difference is that if the environment variable https_proxy isn't set at all then hg just hangs as described above.

The formats for https_proxy I've tried include all variations of:

https_proxy=ip.ip.ip.ip:8070
https_proxy=ip.ip.ip.ip:8080
https_proxy=username:password@ip.ip.ip.ip:8070
https_proxy=username:password@ip.ip.ip.ip:8080
https_proxy=http://ip.ip.ip.ip:8070
https_proxy=http://ip.ip.ip.ip:8080
https_proxy=http://username:password@ip.ip.ip.ip:8070
https_proxy=http://username:password@ip.ip.ip.ip:8080

The results are the same no matter what I set it to.

So the questions I could really use some help with are:

How come I can access my Mercurial repositories at home but not at BitBucket?

How come I can access BitBucket with Git but not with Mercurial using the same configuration?

Does anyone have any ideas of how I can get this to work or what I can test next?

Answer 1

I am also connecting via proxy to bitbucket. As my settings didn't work as expected I found this SO entry.

I noticed that if I use command line parameters then everything works.

hg --config http_proxy.host=192.168.1.1:8080 --config http_proxy.user=Vad1mo --config http_proxy.passwd=secret clone https://bitbucket.org/Vadimo/test

On the other hand same entries in Mercurial.ini didn't work.

[http_proxy]
host = 192.168.1.1
port = 8080
user = Vad1mo
passwd = internet 

By accident I found out the tiny difference between CMD and ini. In CMD the port is postfixed to the host. In ini file it is a new entry.

Changing mercurial.ini to postfix the port to host like on command line solved the problem.

[http_proxy]
host = 192.168.1.1:8080
;port = 8080
user = Vad1mo
passwd = internet 

Maybe this also will help you.

btw. my hg version is 2.6.3

Answer 2

I ran into a similar problem with my work's proxy- actually, almost identical.

I've gotten around the problem so far by setting the http_proxy in mercurial.ini and then connecting to BitBucket via their HTTP address hg.io.

For example, my repository at https://bitbucket.org/mattgwagner/mattgwagner.com can be accessed via http://hg.io/mattgwagner/mattgwagner.com . Of course, this will send your password and connection in plaintext, but at least it let me connect.

This came in more use for me when pulling in open source projects hosted on BitBucket for my use.

Mercurial.ini
[http_proxy] host = 192.168.1.155:8080
no =
user = domainUsername
passwd = pass

Answer 3

Are you able to ssh out? Bitbucket supports ssh access and your proxy won't muck w/ that if it's allowed.