CODEWITHSUNDEEP
javaeclipsemacos
Rob Graeber
Rob Graeber

Reputation: 179

Strange Puzzle - Invalid memory access of location

The error message I'm getting consistently is:

Invalid memory access of location 0x8 rip=0x10cf4ab28

What I'm doing is making a basic stock backtesting system, that is iterating huge arrays of stocks/historical data across various algorithms, using java + eclipse on the latest Mac Os X.

I tracked down the code that seems to be causing it. A method that is used to get the massive arrays of data and is called thousands of times. Nothing is retained so I don't think there is a memory leak. However there seems to be a set limit of around 7000 times I can iterate over it before I get the memory error.

The weird thing is that it works perfectly in debug mode. Does anyone know what debug mode does differently in Eclipse?

Giving the jvm more memory doesn't help, and it appears to work fine using -xint. And again it works perfectly in debug mode.

public static List<Stock> getStockArray(ExchangeType e){
    List<Stock> stockArray = new ArrayList<Stock>();
    if(e == ExchangeType.ALL){
        stockArray.addAll(getStockArray(ExchangeType.NYSE));
        stockArray.addAll(getStockArray(ExchangeType.NASDAQ));
    }else if(e == ExchangeType.ETF){
        stockArray.addAll(etfStockArray);
    }else if(e == ExchangeType.NYSE){
        stockArray.addAll(nyseStockArray);
    }else if(e == ExchangeType.NASDAQ){
        stockArray.addAll(nasdaqStockArray);
    }
    return stockArray;
}

A simple loop like this, iterated over 1000s of times, will cause the memory error. But not in debug mode.

for (Stock stock : StockDatabase.getStockArray(ExchangeType.ETF)) {
    System.out.println(stock.symbol);
}

Upvotes: 4

Views: 1369

Answers (3)

Jesse
Jesse

Reputation: 36

This is probably too late, but for other people who have this problem, this is how you fix it. Instead of using the enhanced for loop, use a normal one. I'm serious.

ex:

for(int stock = 0; stocks < StockDataBase.getStockArray(ExchangeType.ETF).size(); stock++)

You can take it from there.

This is due to the fact that you are creating new objects each time the enhanced for loop loops, but not in the regular for loop. The garbage collector probably doesn't have enough time to address all of those new objects at once.

Upvotes: 2

durron597
durron597

Reputation: 32343

Rob, your code could be far more efficient and use a lot less memory if you use a lot of the suggestions in the comments... which would naturally fix your original problem.

public enum ExchangeType {
    ALL, ETF, NYSE, NASDAQ;

    private static EnumMap<ExchangeType, List<Stock> stocks;

    // Do this during your initialization routine, or if something changes
    public static loadStock(ExchangeType et, List<stock> stockList) {
        stocks.put(et, stockList);
    }

    public List<Stock> getStocks() {
        return stocks.get(this);
    }
}

Then, for security, you could even implement your own ImmutableList...

public class ImmutableList<E> implements List<E> {
    private ArrayList<E> _internal;

    // Do something like this for modification methods
    public boolean add(E e) {
        throw new UnsupportedOperationException("This list is immutable!");
    }

    // Do something like this for access methods
    public E get(int index) {
        return _internal.get(index);
    }
}

This should get your memory under control AND greatly improve performance!!

In any event, if this doesn't fix your problem, as a lot of others said this is probably a JVM bug. As you noted the compile threshhold (i.e. -XX:CompileThreshold=100000) is a quick-and-dirty fix that should solve your problem at least for now...

Upvotes: 2

jlordo
jlordo

Reputation: 37833

Not the best solution, but you say you don't modify the lists, so you could do something like this:

public static List<Stock> getStockArray(ExchangeType e){
    if(e == ExchangeType.ALL){
        List<Stock> stockArray = new ArrayList<Stock>(nyseStockArray.size() + nasdaqStockArray.size());
        stockArray.addAll(nyseStockArray);
        stockArray.addAll(nasdaqStockArray);
        return stockArray;
    }else if(e == ExchangeType.ETF){
        return etfStockArray;
    }else if(e == ExchangeType.NYSE){
        return nyseStockArray;
    }else if(e == ExchangeType.NASDAQ){
        return nasdaqStockArray;
    } else {
       return new ArrayList<Stock>();
}

Upvotes: 0

Related Questions