CODEWITHSUNDEEP
phpapachecron
yura
yura

Reputation: 169

Apache cron: synchronous tasks execution

I want to create an aggregator, which goal is to grabb facebook posts (text, pictures, videos). Next step is to post grabbed info on my resource, and video on my youtube channel. I want to let cron launch this aggregator every 1 minute. But I think, that I will meet next problem: If I grabbed such large video file, it can be not complete uploading to youtube till 1 minute. Till this minute it make happen so, that next video file will be grabbed and it need to be uploaded too. My question: is this second video file will be stay in queue and wait till first will be uploaded, or I need to create a multi threads for this? If so, please tell me how?

Upvotes: 1

Views: 78

Answers (1)

phil-lavin
phil-lavin

Reputation: 1217

Crontab simply spawns a process at a scheduled time. If cron is still running a process that it initiated 1 minute ago, it will create another process (i.e. 2 will be running at the same time).

Your code must ensure that files which are being uploaded are marked such that, if/when cron runs a second process, it doesn't try to upload the same file multiple times.

Your logic would be something like this:

  • Grab data
  • Before you upload that data, check (in a database, for example) if you are already uploading it
  • If you are not already uploading it, mark in the database that you are uploading that data
  • Upload the data

To be honest, this isn't a great use of cron and you'd do better with a long-running process. Software like supervisord can allow you to create a long-running PHP script which is automatically restarted if it crashes.

Upvotes: 1

Related Questions