CODEWITHSUNDEEP
mysqlsqldatabaseindexing
Jacek Francuz
Jacek Francuz

Reputation: 2498

How to avoid full table scan for mutliple searcheable columns query?

I have a following tableA:

id | col1 | col2 | col3 | ... | col66
-------------------------------------
999|  1   |  0   |  0   | ... |  1

All the columns colX are searcheable and there are 66 of them which means that creating efficient index is not possible (at least I think so).

The queries may look like that:

SELECT id FROM tableA WHERE col21=1 AND col31=1 AND col64=1

As you can see I only need to retrieve rows with certain columns being set to '1'. The set of columns may vary. Do you know any way of doing it without full table scan that is time-consuming? I think I have already tried everything but with no luck :-( Thanks!

Upvotes: 0

Views: 148

Answers (2)

mvp
mvp

Reputation: 116407

In order for query like this:

SELECT id FROM tableA WHERE col21=1 AND col31=1 AND col64=1

to work fast, you must have created compound index which includes all fields mentioned: (col21, col31, col64). I guess you cannot predict list of needed fields upfront, so it probably would not work for you - unless you are willing to create huge number of indexes.

Better alternative is to normalize your table - create slave table which will store one attribute in a separate row. This will make indexing much easier, and also makes it possible to keep arbitrary number of attributes.

UPDATE: One more possibility is to replace your 0/1 columns with bitmaps. If today you have one 32-bit primary key and 66 tiny int columns, row width is 4+66=70 bytes. If you use BIT data type (you will need 2 bit columns, becase max BIT size in MySQL is 64 bits), row width becomes 4+(66/8)=13 bytes, or almost 5 times less than before. If today your full scan time is 0.4 seconds, with this packing it would be 0.08 seconds, which might become acceptable for you.

Upvotes: 2

Stephan
Stephan

Reputation: 8090

You can change the structure of your table with something like:

id   type  value
999  col1  1
999  col2  0
---

And you can add a artificial PK if you think you need it also the key thing here is to have a composite index on type and value : INDEX (type,value) this way your queries will use this index and the search will be faster.

Your query will be transformed in :

SELECT 
    id 
FROM 
    tableA 
WHERE 
    type IN ('col21','col31','col64')
    AND value = 1

Upvotes: 1

Related Questions