CODEWITHSUNDEEP
mysqlperformance
getting-there
getting-there

Reputation: 1409

speed up mysql update query

I'm running what I thought would be a fairly straight forward calculation of financial returns on a fairly large (65,000 line) data set. I've run the query on a smaller data set and it works fine, but on the large data set I terminated it after about 20 hours.

The query calculates 3, 6, 12, 18 & 24 month returns for each of the 500 stocks in the universe over about 20 years of monthly data.

UPDATE 
  master_backup AS curr
  LEFT OUTER JOIN
      master AS mo3
      ON 
        ((curr.new_date = LAST_DAY(mo3.new_date - INTERVAL 3 MONTH)) AND (curr.sedol = mo3.sedol))
  LEFT OUTER JOIN 
      master AS mo6
      ON 
        ((curr.new_date = LAST_DAY(mo6.new_date - INTERVAL 6 MONTH)) AND (curr.sedol = mo6.sedol))
  LEFT OUTER JOIN 
      master AS mo12
      ON 
        ((curr.new_date = LAST_DAY(mo12.new_date - INTERVAL 12 MONTH)) AND (curr.sedol = mo12.sedol))
  LEFT OUTER JOIN 
      master AS mo18
      ON 
        ((curr.new_date = LAST_DAY(mo18.new_date - INTERVAL 18 MONTH)) AND (curr.sedol = mo18.sedol))
  LEFT OUTER JOIN 
      master AS mo24  
      ON 
        ((curr.new_date = LAST_DAY(mo24.new_date - INTERVAL 24 MONTH)) AND (curr.sedol = mo24.sedol))
SET 
    curr.ret_3mth = (mo3.price - curr.price)/curr.price,
    curr.ret_6mth = (mo6.price - curr.price)/curr.price,
    curr.ret_12mth = (mo12.price - curr.price)/curr.price,
    curr.ret_18mth = (mo18.price - curr.price)/curr.price,
    curr.ret_24mth = (mo24.price - curr.price)/curr.price;

The table 'master'

'master', 'CREATE TABLE `master` (
  `mdate` date NOT NULL,
  `new_date` date DEFAULT NULL,
  `id_connah` varchar(45) DEFAULT NULL,
  `ticker` varchar(45) NOT NULL,
  `pnum` varchar(45) NOT NULL,
  `sedol` varchar(45) NOT NULL,
  `cusip` varchar(45) NOT NULL,
  `price` double DEFAULT NULL,
  `mcap` double DEFAULT NULL,
  `ret` double DEFAULT NULL,
  `ebit` double DEFAULT NULL,
  `debt_st` double DEFAULT NULL,
  `debt_lt` double DEFAULT NULL,
  `cash` double DEFAULT NULL,
  `assets_total` double DEFAULT NULL,
  `assets_intang` double DEFAULT NULL,
  `assets_curr` double DEFAULT NULL,
  `ev` double DEFAULT NULL,
  `ey` double DEFAULT NULL,
  `roce` double DEFAULT NULL,
  `ret_3mth` double DEFAULT NULL,
  `ret_6mth` double DEFAULT NULL,
  `ret_12mth` double DEFAULT NULL,
  `ret_18mth` double DEFAULT NULL,
  `ret_24mth` double DEFAULT NULL,
  `rank_ey` int(11) DEFAULT NULL,
  `rank_roce` int(11) DEFAULT NULL,
  `rank_combined` int(11) DEFAULT NULL,
  `rank_final` int(11) DEFAULT NULL,
  `fractile` int(11) DEFAULT NULL,
  KEY `sedol` (`sedol`),
  KEY `cusip` (`cusip`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8'

I am by no means a MySQL expert and have set the data base up using the default cnf file settings. Any suggestions would be greatly appreciated. I thought that memory might be an issue, but the process appears more CPU intensive then memory intensive.

Thanks

Upvotes: 2

Views: 697

Answers (1)

Mosty Mostacho
Mosty Mostacho

Reputation: 43434

Have you tried creating indexes on new_date field?

Besides, I think you could also use only 1 LEFT JOIN with the 24 month interval and then update your fields with an if() function based on the conditions you are currently joining those tables.

Upvotes: 1

Related Questions