How to unpublish an iCal (*.ics) feed?

Question

One feature on my site allows registered users to create calendars for their organization. We provide a dynamically-generated iCal feed for these calendars through a URL with query-string parameters. Anyone can subscribe to these feeds by entering the provided URL into Google Calendar, Outlook, iPhone, etc...

This has been working well enough for a few years, but we now have a problem with stale or deleted calendars. If a registered user significantly alters or deletes their account, the calendar will no longer exist and the feed is useless. We currently return a "404 - Not Found" error for those requests (recently changed from "400 - Bad Request").

My question is, other than returning the 404, is there any way to get subscribers to stop requesting a bad feed? This is a similar question, where the accepted answer suggests returning 404 or 410 and hoping the clients will see the error and manually remove the subscription.

That doesn't seem to be working so far. We get ~ 100k feed requests an hour and a full 30% of those are for deleted calendars.

Do Google, Apple, et al not give up when they repeatedly get a 404 for a feed? How have others handled this issue?

If this was just a problem with log pollution I wouldn't worry too much about it. However, since the feeds are dynamically generated, each request hits the backend db. The processing is trivial and doesn't appear to be affecting performance, but the situation can only get worse.

Apologies if this belongs on ServerFault. While the issue affects my servers, I believe the solution is programmatic.

Answer 1

It doesn't look like 410 GONE is any better than 404 NOT FOUND. All the calendar apps hitting our feeds seem to treat either response as "ask again later." (And some of them retry hourly!)

One approach that does seem to help is returning a static "missing calendar," using a 200 response, with:

• a long REFRESH-INTERVAL to discourage calendar app polling (with equivalent X-PUBLISHED-TTL for older apps)
• calendar content prompting the user to clean things up (we use a daily recurring event)

It's not 100% effective: not all apps respect refresh interval, and I'd imagine not all users notice the event (or can figure out how to unsubscribe). But in practice, we're seeing it seems to reduce missing calendar traffic quite a bit compared to the 4xx responses we had before.

Here's an example (you'll want to change at least the PRODID and UID):

BEGIN:VCALENDAR
VERSION:1.0
PRODID:-//Your Company//Your Product v1.0//EN
METHOD:PUBLISH
NAME:Unavailable
X-WR-CALNAME:Unavailable
DESCRIPTION:Calendar not available--please remove it
X-WR-CALDESC:Calendar not available--please remove it
REFRESH-INTERVAL;VALUE=DURATION:P4W
X-PUBLISHED-TTL:P4W
BEGIN:VEVENT
SUMMARY:Calendar unavailable
DTSTART;VALUE=DATE:20240101
DTSTAMP:20240101T000000Z
UID:deleted-20240101@yourproduct.example.com
RRULE:FREQ=DAILY
DESCRIPTION:This calendar is no longer available.\n\n
 Please unsubscribe or remove it from your calendar app settings.
LAST-MODIFIED:20240401T000000Z
END:VEVENT
END:VCALENDAR

Answer 2

I don't believe there is an easy answer - I think it's been asked before.

It's like having to deal with all the traffic when some hackers use your site for target practise on logins or xmlrpc or just looking for vulnerabilities. Or the spammers trying a scatter gun approach sending emails. Or when a web spider decides to excessively crawl your site. You have to size for all that non useful traffic.

You could possibly generate and keep up to date a list outside of the database of the bad ics URLs and have a script check and bounce the request before it gets near the database ? Basically try to deal as efficiently as possible with the problem.

You could also in account deletion try adding a step that requests the user go to their calendar programs and delete the feed before continuing. However that might cause bad vibes and probably would not totally fix it anyway.