CODEWITHSUNDEEP
javaexception
David Koelle
David Koelle

Reputation: 20824

How to know about OutOfMemory or StackOverflow errors ahead of time

In Java, is there a way to know that a StackOverflow error or OutOfMemory exception may happen soon?

The OutOfMemory exception might be an easier one to catch, if one is capable of getting memory usage statistics programmatically, and if one knows ahead of time how much memory needs to be used before the OutOfMemory exception is thrown. But are those values knowable?

For the StackOverflow error, is there a way to get recursion depth, and how does one know what value for recursion depth would cause the error to occur?

By knowing ahead of time whether these errors will happen, I feel I can recover the application more gracefully, instead of watching it crash.

Upvotes: 9

Views: 3857

Answers (11)

Mykhaylo Adamovych
Mykhaylo Adamovych

Reputation: 20966

Implementation example using MemoryPoolMXBean

import static java.lang.management.ManagementFactory.getMemoryMXBean;
import static java.lang.management.ManagementFactory.getMemoryPoolMXBeans;
import static java.lang.management.MemoryNotificationInfo.MEMORY_THRESHOLD_EXCEEDED;
import static java.lang.management.MemoryType.HEAP;
import java.lang.management.MemoryPoolMXBean;
import java.lang.management.MemoryUsage;
import java.util.Collection;
import java.util.concurrent.CopyOnWriteArrayList;
import javax.management.Notification;
import javax.management.NotificationEmitter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class MemoryWatcher {

    public interface Listener {
        void memoryUsageLow(long usedMemory, long maxMemory);
    }

    private static final MemoryPoolMXBean tenuredGenPool = findTenuredGenPool();
    private static final Logger logger = LoggerFactory.getLogger(MemoryWatcher.class);

    private static MemoryPoolMXBean findTenuredGenPool() {
        for (MemoryPoolMXBean pool : getMemoryPoolMXBeans())
            if (pool.getType() == HEAP && pool.isUsageThresholdSupported())
                return pool;
        return null;
    }

    public static MemoryPoolMXBean getTenuredGenPool() {
        return tenuredGenPool;
    }

    private final Collection<Listener> listeners = new CopyOnWriteArrayList<>();

    public MemoryWatcher(double usageThresholdPercent) {
        if (tenuredGenPool == null) {
            logger.warn("Tenured pool is not used");
            return;
        }
        if (tenuredGenPool.getUsageThreshold() != 0)
            logger.warn("Overriding tenured usage threshold {} with {}", tenuredGenPool.getUsage().getMax() / (double) tenuredGenPool.getUsageThreshold(), usageThresholdPercent);
        tenuredGenPool.setUsageThreshold((long) (tenuredGenPool.getUsage().getMax() * usageThresholdPercent));

        NotificationEmitter emitter = (NotificationEmitter) getMemoryMXBean();
        emitter.addNotificationListener((Notification n, Object hb) -> {
            MemoryUsage usage = tenuredGenPool.getUsage();
            if (n.getType().equals(MEMORY_THRESHOLD_EXCEEDED) && usage.getMax() == usage.getCommitted())
                listeners.forEach(listener -> listener.memoryUsageLow(usage.getUsed(), usage.getMax()));
        }, null, null);
    }

    public boolean addListener(Listener listener) {
        return listeners.add(listener);
    }

    public boolean removeListener(Listener listener) {
        return listeners.remove(listener);
    }
}

Upvotes: 0

Clint
Clint

Reputation: 9058

The MemoryMXBean can emit notifications if your memory reaches a certain threshold.

Upvotes: 1

Tom Hawtin - tackline
Tom Hawtin - tackline

Reputation: 147164

One useful thing you can do is use SoftReferences for caches. That will give you a gradual performance slide as you run out of memory. Just don't use WeakReference even in a WeakHashMap because it will annoy me when your application dies nastily on me.

Upvotes: 1

RichN
RichN

Reputation: 6241

For StackOverflowError:

To know the current depth, usually it's either:

  1. using a stateful function (storing the depth in outside the function)
  2. using an accumulator (passing the depth as an argument to the function)

Knowing the depth it will occur is difficult. There are several factors:

  1. The stack space allocated to the JVM (you can change this with the -Xss option)
  2. The amount of stack space already used
  3. The amount used by the current function.

Why not try it out using something like this?

public static void main(String[] args) {
    try {
        recurs();
    } catch (Throwable t) {
        // not a good idea in production code....
    }
}
static int depth = 0;
static void recurs() {
    System.out.println(depth++);
    recurs();
}

Run it several times. Also try adding dummy variables. It can be seen that even the same code may halt at different depths and adding more variables cause it to end earlier. So yeah, pretty much it's unpredictable.

I suppose that besides rewriting the algorithm the only option would be to increase the stack space with the -Xss option.

For OutOfMemoryError, there's the -Xmx option

Upvotes: 0

Elijah
Elijah

Reputation: 13604

Anticipating Out of Memory Errors

I'm surprised I didn't see this mentioned in the other posts, but you can use ManagementFactory in Java 5/6 to get at a lot of the memory usage information.

Look at the platform mbean server page for more information on detecting low memory conditions in Java. I believe you can setup notifiers to call code when memory usage reaches a certain threshold.

Upvotes: 8

Steve B.
Steve B.

Reputation: 57324

Usually memory useage is hard to predict.

Stack Overflows from infinite regression generally show up in the write/debug cycle.

Harder to catch are memory issues on things like large held collections, caches, etc.etc. As has been pointed out you can check runtime free and max memory, but be careful, as their meanings aren't obvious - "free" here, means "free now", e.g. meaning if you ran a full gc you might get more. Unfortunately there's no way to get the "total possible free, including garbage-collectible" without running System.gc(), which is not a nice thing to do on a production application (where you're liable to have large enough data sets to cause the problem in the first place) because the entire JVM will come to a screeching halt for a few seconds (or more, in a large app). Note that even System.gc() is not guaranteed to run "now", but my experience has been that it has whenever I've played with it.

You can print gc activity from the running jvm by starting java with -verbose:gc, -XX:+PrintGCTimeStamps, and -XX:+PrintGCDetails (more detail here), and in general if the collector starts to run more frequently, it's probably a sign that you're running out of memory.

Upvotes: 1

Avi
Avi

Reputation: 20142

You could discover a lot about recursion depth, by creating a Throwable object, and querying its getStackTrace() method. But this is expensive to do.

If you really have a method with a small potential of throwing a StackOverflowError or an OutOfMemoryError why not just insert try-catch blocks and catch those errors? They can be caught and handled just like checked exceptions.

Upvotes: 0

Robin
Robin

Reputation: 2616

I don't know anything about working this out at run time, or what you might be able to do to avoid it once you predict it is going to happen. Better to try and avoid it occurring in the first place.

1) You could use Findbugs which may indicate some StackOverFlow errors occurring from inadvertently calling the same method from itself.

2) You could store data likely to cause you to run out of memory with a SoftReference and have a null check on accessing it so it can be reloaded if it's been garbage collected.

If either of these things are actually issues for you then the solution probably isn't in detecting it happening but architecting your solution differently to avoid them occuring if at all possible.

Upvotes: 0

gustafc
gustafc

Reputation: 28865

You can anticipate out-of-memory conditions with Runtime.freeMemory() and Runtime.maxMemory(). Most times it'll be hard recovering gracefully, but I leave that to you.

Upvotes: 5

Uri
Uri

Reputation: 89749

Most StackOverflow errors come out of bad recursion. Unfortunately, the problem of determining if a recursion will stop is generally not decidable (this is a central concept in CS). There are cases, however, you could get warnings, for example some IDEs will let you know if you're invoking a function recursively with no parameters.

Upvotes: 3

John Topley
John Topley

Reputation: 115372

You should never see a StackOverflow exception if your application is designed and implemented correctly!

Generally, if you get a StackOverflow exception then it's a sign that there's a bug in your recursion code.

Upvotes: 2

Related Questions