CODEWITHSUNDEEP
performancesurvey
Robert Gould
Robert Gould

Reputation: 69855

How much a performance boost is worthy/notable?

Everyone has their views on when and where to increase performance, so that's not of interest here, but I was wondering at what point a performance gain becomes worthy of promotion (say from branch development into the trunk, or enough to be placed on some progress report or whatever).

For example my rule-of-thumb is a 10% performance boost is noteworthy, but a 5% is nothing special (of course added up they may be worthy though)

Note: as there is no correct answer marking this wiki, but I do feel having opinions on the subject is useful when making decision about performance

Upvotes: 0

Views: 196

Answers (8)

innaM
innaM

Reputation: 47839

I'd rather use absolute values. 20% of 1 ms isn't something you care about. But 2% of an hour is pretty impressive.

Upvotes: 4

Jonas Kölker
Jonas Kölker

Reputation: 7837

Here's a simple test: give someone the unoptimized system and ask them to use it. Give them the optimized system and ask them to use it. Ask them if they can tell the difference, and which they like best. (be sure that it's a double-blind test and that the systems are otherwise equal). If people like the optimized system more, it's worth making noise about.

Here's a simpler test: can you tell the difference? If you can't, then it's not worth mentioning. (on the other hand, small improvements accumulate).

Upvotes: 2

Garry Shutler
Garry Shutler

Reputation: 32698

Basically when it is an order of magnitude or better for a complete unit of work.

  • Hours to minutes batch processes
  • Minutes to seconds user triggered batch processes, installs, builds
  • Seconds to a second user triggered processes
  • A second to instant user triggered processes

Upvotes: 1

Hexagon
Hexagon

Reputation: 6961

It depends on context.

In an application that needs to play video in real-time, a 1ms improvement per frame can be the difference between 30 frames per second (which is a common rate for video clips) to 29 frames per second (which would look - visually - much worse due to dropping frames).

In another application that runs over-night, two hours vs. three hours may not make any real world difference (but the load caused to the database while running can be very significant - optimize that instead!).

Upvotes: 2

sharptooth
sharptooth

Reputation: 170489

If increased performance adds a new quality to the system, it's worth mentioning. Otherwise noone usually cares.

For example, if the system used to work noticeably slow when loaded with a reasonable but above usual amount of data and you've fixed this problem it's worth mentioning because those users who load huge amounts of data will benefit from the improvement.

Upvotes: 3

starblue
starblue

Reputation: 56772

It depends.

If those 5% translate into money saved e.g. on a big server farm it would be noteworthy.

If it is 5% faster on an end user's computer, who wouldn't even notice, then not.

Upvotes: 2

BlueMonkMN
BlueMonkMN

Reputation: 25601

I think it depends on the elegance/readability of the code too. If a particular change feels like a more correct solution, eliminating some stupidity in the code then if it also improves performance at all, it's "worthy". However, if a particular change is designed purely for performance, and decreases code readability or maintainability, then it better be a pretty good boost to make it worthy... maybe 50%.

Upvotes: 2

Anton Gogolev
Anton Gogolev

Reputation: 115749

O(n) to O(log n) algorithm improvement for big n's is very good.

Upvotes: 2

Related Questions