CODEWITHSUNDEEP
pythonhtmlxpathsec
noelyh
noelyh

Reputation: 11

Xpath query trying to pull a value from a website table

I am trying to make a loop to go into a list of company profiles on sec.report and gather their "state of Incorporation" but only getting blank results back.

For example, on https://sec.report/Ticker/adbe, I am trying to use the Xpath function to pull out the value "Delaware". I am using the following code, with an xpath that i have double checked on the developer view of google chrome: sec = tree.xpath('//body/div[1]/div/div[3]/div[2]/table/tbody/tr[3]/text()')

Can someone please show me where i have gone wrong here? is it due to the "state of incorporation" not being being a defined class?

I have also pulled the CIK number for each company but these values are coming through successfully using a very similar code so i'm not too sure where i have gone wrong: sec = tree.xpath('//div[1]/div/h2[1]/text()')

Upvotes: 0

Views: 134

Answers (1)

Dazed
Dazed

Reputation: 1551

your first one would have worked if you used the below but those are brittle xpaths. Hit this site and it will teach you how to write better paths - https://www.w3schools.com/xml/xpath_intro.asp

//body/div[1]/div/div[3]/div[2]/table/tbody/tr[3]/td[2]

You can try these:

Delaware:

//div[@class='panel panel-default'][1]/following::table[1]//td[text()='State of Incorporation']/following-sibling::td

CIK:

//h2[contains(text(), 'SEC')]

Upvotes: 0

Related Questions