CODEWITHSUNDEEP
arraysperformancegoslice
Pierre Prinetti
Pierre Prinetti

Reputation: 9622

Do I have to avoid appending for performance?

I am new to Golang.
Should I always avoid appending slices?

I need to load a linebreak-separated data file in memory. With performance in mind, should I count lines, then load all the data in a predefined length array, or can I just append lines to a slice?

Upvotes: 2

Views: 1796

Answers (1)

Volker
Volker

Reputation: 42413

You should stop thinking about performance and start measuring what the actual bottleneck of you application is.

Any advice to a question like "Should do/avoid X because of performance?" is useless in 50% of the cases and counterproductive in 25%.

There are a few really general advices like "do not needlessly generate garbage" but your question cannot be answered as this depends a lot on the size of your file:

  • Your file is ~ 3 Tera byte? Most probably you will have to read it line by line anyway...
  • Your file has just a bunch (~50) of lines: Probably counting lines first is more work than reallocating a []string slice 4 times (or 0 times you you make([]string,0,100) it initially). A string is just 2 words.
  • Your file has an unknown but large (>10k) lines: Maybe it might be worth. "Maybe" in the sense you should measure on real data.
  • Your file is known to be big (>500k lines): Definitively count first, but you might start hitting the problem from the first bullet point.

You see: A general advice for performance is a bad advice so I won't give one.

Upvotes: 11

Related Questions