CODEWITHSUNDEEP
scalamemory-profiling
lllllllllllll
lllllllllllll

Reputation: 9110

Measure the function-level memory usage in Scala in my Application Code during runtime

First of all, this is not an "off-line" profiling task!

I am working on some SCala codebase, and currently what I am trying to do is, if a function foo consumes too many memory (let's say over 10G), kill this function and return a default value.

So it should look like:

monitor{
  foo()   <--- if foo has used over 10G memory, just cut it off
}
catch {
  case MemoryUsageError => default_value
}

Note that currently foo is running in the same process with my main function.

Is it possible to do so? I quickly googled such materials and only find a way to show the current memory usage of a SCala application; it is not as fine-grained as what I am looking for.

Am I clear on this? Could anyone shed some lights here? Thanks a lot!

========================================================================

Note that what I am looking for is an "online" method! It is not like off-line profiling. My application ifself should determine the memory usage of foo function, and if it goes too high, just cut it off.

Upvotes: 2

Views: 1339

Answers (1)

Zernike
Zernike

Reputation: 1766

Is it possible?

In general jvm doesn't track creator of objects allocated on heap and place of creation. This is very costly and doesn't matter for GC.

How to live with it

Termination

  1. Self-controlled program. If you want terminate some continuous computation then computation shouldn't be continuous. What you need is check points where condition could be validated. For example, every start of iteration in a loop or at the beginning of every recursive call. Obviously computation could consist of several different stages instead of simple loop but approach is the same.
  2. Separation of computation and control. For example, execute function as Future with predetermined Thread and interrupt it if needed or using ForkJoinTask and cancel() method.

Measurement

  1. Usually only one or couple of classes fulfill most of the memory. If instances are about the same size then memory control could be implemented with counter of objects. Classes of 'heavy' objects could be find by inspection of algorithm or using jvisualvm. Increase counter during instance creation. Decrement is harder. Update counter when references are released (count instances that couldn't be removed by GC) or use PhantomReference (count all instances existed in VM). But don't use finalize()!
  2. Second method is java instrumentation package. It allows to measure objects size (probably there are methods determining consumption of all objects of certain class). Also you could try measuring available memory. The flaw is you measure objects of not certain function but all of them.
  3. For time control write down timestamp at the beginning of computation and measure duration at every check point.

Upvotes: 0

Related Questions