CODEWITHSUNDEEP
phpsqlperformance
Cam Jackson
Cam Jackson

Reputation: 12294

Getting all data in one query and processing in PHP vs. querying repetitively for each piece of data

I know ahead of time that I'm eventually going to need a bunch of rows from a certain table, but I don't know in what order I'll need each one.

In general, from a performance and best practices point of view, is it a better idea to:

  1. Do a single general query that returns all the rows I know I'll eventually need, and then use PHP (or whatever) to pick out each one as I need it; or,

  2. Do lots of queries using WHERE to pick out only each row as I need it.

The number of rows I'll need is something like 300-600, and the page shouldn't be getting loaded more than a few times per minute (although I'd like to pick a solution that will cope under heavier loads than this).

Edit for more info: I'm using PostgreSQL. The table is going to get quite large over time, probably with many thousands of rows. I can determine before I do any queries exactly which rows I will eventually need (as I said, n ~ 500), but what I don't know is what order I'll need them in. So it's either 1 query and n PHP searches for specific values in an array, or n queries with a different WHERE clause each time. I'm leaning towards the former.

Upvotes: 1

Views: 1505

Answers (2)

Michael Berkowski
Michael Berkowski

Reputation: 270609

It's nearly always faster to do one query instead of n queries when possible. If your result set is in an associative array, individual rows won't be indexed in the array. But, if you iterate over the result array and key the array elements by an row ID, they will become faster for random access in PHP (rather than having to traverse the whole array each time to find the row you need). Of course, doing so will use a bit more memory to copy the array elements.

Either way, this is much faster than returning to the database each time, especially if you will hit it hundreds of times.

The best option is to pull your records from the RDBMS in the order which you'll need them with ORDER BY. Then random access isn't an issue.

In any case, 600 rows doesn't sound like that much. Make sure you have indexes on appropriate columns so your query is optimized and you'll be fine.

EDIT When fetching rows, create an array index. Now you don't need to search the array, as you can just access it by index.

$rowset = array();
while ($row = pg_fetch_assoc($result)) {
  // Append rows to $rowset indexed by the 'id' column
  // Use whatever column you'll need to be searching with PHP
  $rowset[$row['id']] = $row;
}

Upvotes: 3

k_b
k_b

Reputation: 2480

Build yourself a dynamic SQL query with the appropriate filter in the WHERE clause. You shouldn't be pulling a worst case 600 rows across the wire and into memory - that's not good design.

Upvotes: 0

Related Questions