CODEWITHSUNDEEP
gomemory-management
sup
sup

Reputation: 105

How could I make a Go program use more memory? Is that recommended?

I'm looking for option something similar to -Xmx in Java, that is to assign maximum runtime memory that my Go application can utilise. Was checking the runtime , but not entirely if that is the way to go.

I tried setting something like this with func SetMaxStack(), (likely very stupid)

debug.SetMaxStack(5000000000) // bytes
model.ExcelCreator()

The reason why I am looking to do this is because currently there is ample amount of RAM available but the application won't consume more than 4-6% , I might be wrong here but it could be forcing GC to happen much faster than needed leading to performance issue.

What I'm doing

Getting large dataset from RDBMS system , processing it to write out in excel.

Another reason why I am looking for such an option is to limit the maximum usage of RAM on the server where it will be ultimately deployed.

Any hints on this would greatly appreciated.

Upvotes: 2

Views: 4524

Answers (1)

kostix
kostix

Reputation: 55443

The current stable Go (1.10) has only a single knob which may be used to trade memory for lower CPU usage by the garbage collection the Go runtime performs. This knob is called GOGC, and its description reads

The GOGC variable sets the initial garbage collection target percentage. A collection is triggered when the ratio of freshly allocated data to live data remaining after the previous collection reaches this percentage. The default is GOGC=100. Setting GOGC=off disables the garbage collector entirely. The runtime/debug package's SetGCPercent function allows changing this percentage at run time. See https://golang.org/pkg/runtime/debug/#SetGCPercent.

So basically setting it to 200 would supposedly double the amount of memory the Go runtime of your running process may use.

Having said that I'd note that the Go runtime actually tries to adjust the behaviour of its garbage collector to the workload of your running program and the CPU processing power at hand. I mean, that normally there's nothing wrong with your program not consuming lots of RAM—if the collector happens to sweep the garbage fast enough without hampering the performance in a significant way, I see no reason to worry about: the Go's GC is one of the points of the most intense fine-tuning in the runtime, and works very good in fact.

Hence you may try to take another route:

  • Profile memory allocations of your program. Analyze the profile and try to figure out where the hot spots are, and whether (and how) they can be optimized.

    You might start here and continue with the gazillion other intros to this stuff.

  • Optimize. Typically this amounts to making certain buffers reusable across different calls to the same function(s) consuming them, preallocating slices instead of growing them gradually, using sync.Pool where deemed useful etc.

    Such measures may actually increase the memory truly used (that is, by live objects—as opposed to garbage) but it may lower the pressure on the GC.

Upvotes: 4

Related Questions