CODEWITHSUNDEEP
rest
tpow
tpow

Reputation: 7884

Handling long queries without violating REST

We have a REST api, and we've done a pretty good job at sticking to the spirit of REST. However, we have an important consumer, and they're requesting a way to reconcile their datastore. The flow works like this:

  1. Consumer makes a GET call to retrieve all inventory objects created within a date range. Lets say this returns 1 million inventory VINs.
  2. Consumer compares the payload with their own datastore, see's that they're missing 5,000 inventory objects
  3. Consumer would like to make a request with the 5,000 VIN id's, and return those 5,000 objects.

The problem is that the long query string (JSON array of vins) bumps into the query string length limits imposed by our server. Possbile ideas - make 5k separate calls (seems horrible), increase querystring length limit on server (would like not to do this), use POST instead (not RESTful?).

So, I'm wondering what Roy Fielding would do...

Upvotes: 2

Views: 311

Answers (4)

Will Hartung
Will Hartung

Reputation: 118764

Somewhat similar to what moonwave99 suggested, but instead you create a resource called a "set".

You POST to /set a list of identifiers that you wish to be in the set. The result of the POST is a redirect URL to the resource that names the specific set.

So:

POST /set

Result:

301 Moved Permanently
Location: /set/123

Then:

GET /set/123

Returns the list of items in the set.

Sets are orthogonal to the use case of "fetching differences", they're simply a compilation of items.

If the creation of a set takes a long time, and you consider the set itself to be a snapshot of the data, when the user tries to do the GET /set/123 can simply reply with a 202 Accepted until the actual dataset has been completed.

You can then use:

GET /set/123/identifiers

To get a collection of the actual identifiers in the set, for example, if you like.

You can do something like

POST /setfromquery

and send a list of criteria (name like "John*", city = "Los Angeles", etc.). This doesn't really need its own specific resource, just define your query "language" to include both simple lists of IDs as well as perhaps other filter criteria.

Set operations (unions, differences, etc.). Lots of powerful things can be done with a set resource.

Finally, of course, there's the ever popular:

DELETE /set/123

Upvotes: 2

tpow
tpow

Reputation: 7884

Using a post without creating a resource just seemed too dirty for me. In the end, we made it so that there was a limit of 100 ids requested in a "chunk". In practice, these requests will rarely be > 100, so hacking REST principles to accomodate an edge case seemed like a bad idea. I made sure the limitation was clearly defined in our API docs, done and done...

Upvotes: 0

Martin
Martin

Reputation: 16300

I don't think anyone would fault you in working around GET not accepting a request body by using POST for a request that needs a request body. You are just being pragmatic.

I agree, making 5000 individual requests or upping the query string limit are ugly. POST is the way forward.

Upvotes: 1

moonwave99
moonwave99

Reputation: 22810

What about a POST submitting the JSON file with the id's list to a new resource, e.g. called /inventory/difference?

If the computation goes any long, you can answer with 202 Accepted and the id of the resource being generated, then point back to it at /inventory/difference/:id.

Upvotes: 3

Related Questions