CODEWITHSUNDEEP
webautomation
ilj
ilj

Reputation: 869

Simple web crawler / scraper for deals

For fun and maybe for profit, i want to implement the following:

  1. scheduled or manually triggered process that logs into all my bank accounts
  2. process knows bank site structure, and goes through "cashback/partner deals" pages
  3. all deal information is collected in one place and in one format
  4. when i'm going to buy something, i can quickly see if any of my cards has a special offer for that place. so that i can pay with the card that offers the best deal. ideally, this should happen on my android phone. that doesn't imply a standalone app though, e.g. you can search inside an email or googledoc or anything.

any ideas on implementation? don't limit yourself. suggest crazy things, as long as they work.

PS i did look for an existing website that offers something similar, but they all seem to focus on cards themselves rather than on specific deals for stores.

Upvotes: 1

Views: 350

Answers (1)

davejagoda
davejagoda

Reputation: 2528

Depending on the bank website, your code may need to be able to execute JavaScript to interact with it. Take a look at CasperJS for the web scraping part.

Upvotes: 1

Related Questions