CODEWITHSUNDEEP
goweb-scraping
Lucas Tomic
Lucas Tomic

Reputation: 152

How do I access to a dynamic HTML element with web scrapping?

I'm using go-rod for web scrapping. I want to access to a link which is inside a dynamic <a>. To make this a visible I have to complete a searcher which is an input with the next format (without a submit):

<form>
    <input> <!--This is the searcher-->
<form/>

So, when I complete it, the a I want to access appears: Screenshot of the web

Until here, all It's ok. This is the code I use to complete the searcher:

//Page's url
page := rod.New().MustConnect().MustPage("https://www.sofascore.com/")

//Acept cookies alert
page.MustElement("cookiesAlertSelector...").MustClick()

//Completes the searcher
el := page.MustElement(`searcherSelector...`)
el.MustInput("Lionel Messi")

Now the problem appears, when I want to click the a which has been displayed after I completed the searcher.

I tried with this:

divIWant := page.MustElement("aSelector...")
divIWant.MustClick()

and with this:

divIWant := page.MustElement("aSelector...").MustWaitVisible()
divIWant.MustClick()

But, both of them returns me the same error:

panic: {-32000 Node is detached from document }

goroutine 1 [running]: github.com/go-rod/rod/lib/utils.glob..func2({0x100742dc0?, 0x140002bad50?}) /Users/lucastomicbenitez/go/pkg/mod/github.com/go-rod/[email protected]/lib/utils/utils.go:65 +0x24 github.com/go-rod/rod.genE.func1({0x14000281ca0?, 0x1003a98b7?, 0x4?}) /Users/lucastomicbenitez/go/pkg/mod/github.com/go-rod/[email protected]/must.go:36 +0x64 github.com/go-rod/rod.(*Element).MustClick(0x14000289320) /Users/lucastomicbenitez/go/pkg/mod/github.com/go-rod/[email protected]/must.go:729 +0x9c main.main() /Users/lucastomicbenitez/development/golang/evolutionaryAlgorithm/main/main.go:22 +0x9c exit status 2

So, looking for some solution I found this github issue and tried this to get the link:

link := page.MustEval(`()=> document.querySelector('aSelector...').href`)

But it returns this:

panic: eval js error: TypeError: Cannot read properties of null (reading 'href')

However, I'm pretty sure the selector is correct. What am I doing wrong?

Upvotes: 0

Views: 943

Answers (1)

Lucas Tomic
Lucas Tomic

Reputation: 152

As @Hymns For Disco said in the comments, I just needed to wait some time after the searcher is completed.

el.MustInput("Lionel Messi")

time.Sleep(time.Second)

link := page.MustEval(`()=> document.querySelector('aSelector...').href`)

Upvotes: 2

Related Questions