CODEWITHSUNDEEP
phppythonapiserver
Jonathan Sanchez
Jonathan Sanchez

Reputation: 33

Project using Python Webscraping

I'm working on a project and I just wanted to know what route (If I'm not going down the correct route) I should go down in order to successfully complete this project.

Ferry Time: This application uses boat coordinates and ETA algorithms to determine when a boat will be approaching a certain destination.

Now, I've worked up a prototype that works, but not the way I want it to. In order for my ETA's to show up on my website accurately, I have a python script that runs every minute to webscrape these coordinates from a specific site, do an algorithm, and spit out an ETA. The ETA is then sent to my database, where I display it on my website using PHP and SQL.

The ETA's only update as long as this script is running (I literally run the script on Eclipse and just leave it there)

Now my question: Is there a way I can avoid running the script? Almost like an API. Or to not use a database at all?

Thanks !

Upvotes: 0

Views: 86

Answers (2)

Wells Burke
Wells Burke

Reputation: 26

If the result of your algorithm only depends on the LAST scrape and not a history of several scrapes, then you could just scrape "on demand" and deploy your algo as an AWS lambda function.

Upvotes: 1

lpiner
lpiner

Reputation: 465

Even if they had an API you'd still need to run something against it to get the results. If you don't want to leave your IDE open you could use cron to call your script.

Upvotes: 0

Related Questions