CODEWITHSUNDEEP
pythonselenium-webdriverbrowsermob-proxyhar
Moshe
Moshe

Reputation: 521

How do i get headers from Browsermob-proxy?

What i need : Proxy server log file of a session that includes content/text of the traffic/responses. What i have : Working silenium webdriver and browsermobproxy server log (*but without headers!).

browsermob-proxy working on localhost:8080/proxy/8081

Partial Code:

bmp_path = "path-to-browsermob-proxy.bat"
server = Server(path=bmp_path)
server.start()
proxy_server = server.create_proxy()
proxy_server.new_har()
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server={}'.format(proxy_server.proxy))
browser = webdriver.Chrome("path-to-chromedriver.exe",chrome_options=chrome_options)
browser.get('www.demowebsite.com')

When printing:

pprint(proxy_server.har)

{"log":{"version":"1.2","creator":{"name":"BrowserMob Proxy","version":"2.1.4","comment":""},"pages":[{"id":"Page 0","startedDateTime":"2019-10-03T15:17:20.385+03:00","title":"Page 0","pageTimings":{"comment":""},"comment":""}],"entries":[],"comment":""}}

So far so good, Now here is the thing.

When i add any param to new_har like:

proxy_server.new_har(options={'captureHeaders': True})

Or:

proxy_server.new_har('test') # for ref param

Always get a 500 response from the server:

r = requests.put('%s/proxy/%s/har' % (self.host, self.port), payload)

Problem accessing /proxy/8081/har. Reason: Server Error

{JSONDecodeError}Expecting value: line 1 column 1 (char 0)

It looks like i can't make a request to the proxy server ? Thanks for helpers

Upvotes: 0

Views: 1961

Answers (2)

Celso Jr
Celso Jr

Reputation: 295

Try to clear the DNS cache associated with the proxy instance

proxy_server.clear_dns_cache()

Upvotes: 0

Waller511
Waller511

Reputation: 29

This probably will not work entirely, I'm facing the same type of issue. This is what I did:

rput = requests.put('http://localhost:8080/proxy/8082/har',headers={"Content-Type":"application/json"},data={"captureHeaders":True,"captureCookies":True,"captureContent":True})

It looks like Browsermob proxy doesn't like the params argument to used. Instead, try to send your request with data.

Note I still am not able to get all the headers and cookies, if you figure it out I'd appreciate it if you let me know :)

Upvotes: 1

Related Questions