CODEWITHSUNDEEP
rssweb-crawlerkimono
Andrew Shtern
Andrew Shtern

Reputation: 11

Kimono Labs API suddenly stops working

I've created API with Kimono Labs, to generate RSS feed from a website. It is working ok, crawling data every hour, however every several days it just stop working. No errors, nothing. It the crawl history i can see, that previous crawls was successful, and then API just stop crawling the data. Until i launch manual crawl. Then API start working again, but only for a several days. And then all again, it stops, i initiate manual crawl, it's working for some time. What can cause such a behavior?

Upvotes: 1

Views: 237

Answers (2)

Andrew Shtern
Andrew Shtern

Reputation: 11

Anyway, it was a kimono issue, that is fixed now. I've got an e-mail from support

This is a crawling bug that we've now implemented handling for.
We are running a script that will check for queued scheduled crawls every hour
and start them if they are not running.

Upvotes: 0

AndriuZ
AndriuZ

Reputation: 656

It's intended behaviour described in every API's under (?) popover:

<p>Auto-run frequency <span class="icon-question-circle" data-html="true" popover="Specify how often this API will automatically fetch new data from the target page(s). APIs are limited to 1 URL for a hourly auto-run, <1000 URLs for a daily auto-run, and <10,000 URLs for a weekly auto-run."></span></p>

Upvotes: 0

Related Questions