CODEWITHSUNDEEP
mysqldatabasedatabase-designddldml
r.bhardwaj
r.bhardwaj

Reputation: 1613

Mysql partitioning effect on DDL and DML

I am using Mysql 5.6 with ~150 million records in Transaction table (InnodB). As the size is increasing this table is becoming unmanageable (adding column or index) and slow even with required indexing. After searching through internet I found it is appropriate time to partition the table. I am confidant that partitioning will solve following purpose for me

  1. Improve DML statements response time (using partitioning pruning)
  2. Improve archival process

But I am not sure wether (and how) it will improve DDL performance for this table or not. More specifically following DDL's performance.

  1. ALTER TABLE ADD/DROP COLUMN
  2. ALTER TABLE ADD/DROP INDEX

I went through Mysql documentation and internet but unable to find my answer. Can anyone please help me in this or provide any relevant documentation for this.

My table structure is as following

CREATE TABLE `TRANSACTION` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `parent_id` int(11) DEFAULT NULL,
  `parent_uuid` char(36) DEFAULT NULL,
  `order_number` varchar(64) DEFAULT NULL,
  `order_id` int(11) DEFAULT NULL,
  `order_uuid` char(36) DEFAULT NULL,
  `order_type` char(1) DEFAULT NULL,
  `business_id` int(11) DEFAULT NULL,
  `store_id` int(11) DEFAULT NULL,
  `store_device_id` int(11) DEFAULT NULL,
  `source` char(1) DEFAULT NULL COMMENT 'instore, online, order_ahead, etc',
  `created_at` timestamp NULL DEFAULT NULL,
  `updated_at` timestamp NULL DEFAULT NULL,
  `flags` int(11) DEFAULT NULL,
  `customer_lang` char(2) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `parent_id` (`parent_id`),
  KEY `business_id` (`business_id`,`store_id`,`store_device_id`),
  KEY `parent_uuid` (`parent_uuid`),
  KEY `order_uuid` (`order_uuid`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

And I am partitioning using following statement.

ALTER TABLE TRANSACTION PARTITION BY RANGE (id)
(PARTITION p0 VALUES LESS THAN (5000000) ENGINE = InnoDB,
 PARTITION p1 VALUES LESS THAN (10000000) ENGINE = InnoDB,
 PARTITION p2 VALUES LESS THAN MAXVALUE ENGINE = InnoDB)

Thanks!

Upvotes: 0

Views: 544

Answers (1)

Rick James
Rick James

Reputation: 142208

Partitioning is not a performance panacea. Even the items you mentioned will not speed up; they may even slow down.

Instead, I will critique the table to look for ways to speed up some things.

  • UUIDs are terrible for performance once the index on it becomes too big to be cached. This is because of its randomness. Possible solutions: compact it into BINARY(16); shrink the table other ways; avoid UUIDs.
  • Why have both parent_id and parent_uuid??
  • Shrink the 4-byte INTs to smaller datatypes where practical.
  • Usually CHAR should be CHARACTER SET ascii (1-byte/character), not utf8mb4 (4 bytes/char).
  • Caution: 150M is getting remotely close to the 2-billion limit of INT SIGNED. Consider 4B limit of INT UNSIGNED. (Each is 4 bytes.)
  • Do you ever use created_at or updated_at?
  • MySQL 8.0.13 has a very fast ADD COLUMN and DROP COLUMN (for limited situations).
  • 5.7.?? has a less-invasive ADD INDEX than previous versions, but I am not sure it applies to partitioned tables.
  • 5.7.4: Online DDL support reduces table rebuild time and permits concurrent DML, which helps reduce user application downtime. For additional information, see Overview of Online DDL.

More importantly, let's see the main queries that are "too slow". There may be composite indexes and/or reformulations of the queries that will speed them up.

There is even a slim chance that partitioning will help but not on the PRIMARY KEY.

I think there are only 4 use cases where partitioning helps performance.

Upvotes: 3

Related Questions