CODEWITHSUNDEEP
pythonseleniumautomationweb-scrapingcapybara
bluemihai
bluemihai

Reputation: 454

Automating web tasks?

I play on chess.com and I'd like to download a history of my games. Unfortunately, they don't make it easy: I can access 100 pages of 50 games one at a time, click "Select All" and "Download" and then they e-mail it to me.

Is there a way to write a script, in python or another language, that helps me automate any part of the process? Something that simulates clicking a link? Is Capybara useful for things like this outside of unit testing? Selenium?

I don't have much experience with web development yet. Thanks for your help!

Upvotes: 2

Views: 431

Answers (3)

Tytus
Tytus

Reputation: 648

Take a look at Helium. It should be relatively easy to achieve what you need to do using this Python library:

start_chrome("chess.com")
...
click("Select All")
click("Download")
...

etc.

Upvotes: 0

ddavison
ddavison

Reputation: 29092

Selenium could be a good candidate if you want to do something like this.

Here would be some pseudo code:

foreach page_link in page_links:
  page_link.click()
  select_all.click()
  download_link.click()

Upvotes: 0

Chris Hawkes
Chris Hawkes

Reputation: 12430

You may want to check out CasperJS. I use Python to fire CasperJS scripts to do web scraping and return data to Python to parse further or store to a database etc...

Python itself has BeautifulSoup and Mechanize but the combination is not great with Ajax based sites.

Python and CasperJS is perfect.

Upvotes: 1

Related Questions