CODEWITHSUNDEEP
neo4jspring-data-neo4j
Ryan Walls
Ryan Walls

Reputation: 7092

Embedded Neo4j database throws permgen and OutOfMemoryError when starting

I'm using Spring Data Neo4j 3.2.0 and Neo4j 2.1.2.

I'm using Gradle as my build tool and starting the application in development using the gradle tomcat plugin. For days this was working fine.

Now in the past day, I'm starting getting memory errors when I start the application. Sometimes the errors don't happen until I run a query, but occasionally I get memory errors just on startup. 

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1102557897-37"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1102557897-36"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "HashSessionScavenger-0"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1102557897-38"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1102557897-39"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1102557897-40"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "RMI RenewClean-[10.30.65.86:51134]"

How I'm starting the embedded server:

public EmbeddedGraphDatabase createEmbeddedDatabase(String storeDir) {
        return (EmbeddedGraphDatabase) new GraphDatabaseFactory()
                .newEmbeddedDatabaseBuilder( storeDir )
                .setConfig(GraphDatabaseSettings.allow_store_upgrade, "true")
                .setConfig(ShellSettings.remote_shell_enabled, "true")
                .newGraphDatabase();

Gradle is being started with standard config which looks like this once running:

java -Xmx512m -classpath /Users/rwalls/Development/gradle-1.12/lib/gradle-launcher-1.12.jar org.gradle.launcher.GradleMain tomcatRunWar

Store file sizes:

{
    "description": "Information about the sizes of the different parts of the Neo4j graph store",
    "name": "org.neo4j:instance=kernel#0,name=Store file sizes",
    "attributes": [
      {
        "description": "The total disk space used by this Neo4j instance, in bytes.",
        "name": "TotalStoreSize",
        "value": 291941310,
        "isReadable": "true",
        "type": "long",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "The amount of disk space used by the current Neo4j logical log, in bytes.",
        "name": "LogicalLogSize",
        "value": 184,
        "isReadable": "true",
        "type": "long",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "The amount of disk space used to store array properties, in bytes.",
        "name": "ArrayStoreSize",
        "value": 128,
        "isReadable": "true",
        "type": "long",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "The amount of disk space used to store nodes, in bytes.",
        "name": "NodeStoreSize",
        "value": 4341060,
        "isReadable": "true",
        "type": "long",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "The amount of disk space used to store properties (excluding string values and array values), in bytes.",
        "name": "PropertyStoreSize",
        "value": 16972278,
        "isReadable": "true",
        "type": "long",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "The amount of disk space used to store relationships, in bytes.",
        "name": "RelationshipStoreSize",
        "value": 39276426,
        "isReadable": "true",
        "type": "long",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "The amount of disk space used to store string properties, in bytes.",
        "name": "StringStoreSize",
        "value": 1476352,
        "isReadable": "true",
        "type": "long",
        "isWriteable": "false ",
        "isIs": "false "
      }
    ],
    "url": "org.neo4j/instance%3Dkernel%230%2Cname%3DStore+file+sizes"
  }

Server config (which seems to be startup dependent):

{
    "description": "The configuration parameters used to configure Neo4j",
    "name": "org.neo4j:instance=kernel#0,name=Configuration",
    "attributes": [
      {
        "description": "Configuration attribute",
        "name": "store_dir",
        "value": "/Users/rwalls/Development/neo4j-community-2.1.2/data/graph.db",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "Configuration attribute",
        "name": "neostore.nodestore.db.mapped_memory",
        "value": "460M",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "Enable a remote shell server which shell clients can log in to",
        "name": "remote_shell_enabled",
        "value": "true",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "Configuration attribute",
        "name": "ephemeral",
        "value": "false",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "Configuration attribute",
        "name": "neostore.propertystore.db.strings.mapped_memory",
        "value": "1408M",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "Configuration attribute",
        "name": "neostore.propertystore.db.arrays.mapped_memory",
        "value": "1596M",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "Configuration attribute",
        "name": "allow_store_upgrade",
        "value": "true",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "Configuration attribute",
        "name": "neostore.relationshipstore.db.mapped_memory",
        "value": "2031M",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      },
      {
        "description": "Configuration attribute",
        "name": "neostore.propertystore.db.mapped_memory",
        "value": "1657M",
        "isReadable": "true",
        "type": "java.lang.String",
        "isWriteable": "false ",
        "isIs": "false "
      }
    ],
    "url": "org.neo4j/instance%3Dkernel%230%2Cname%3DConfiguration"
  }

I realize I could start java with more memory allocated... but tried that with many different options and still had the errors.

Ideas?

My next step is to run the application in a standard tomcat instance (instead of from gradle) to see if that makes a difference.

EDIT

Stack trace:

Caused by: java.lang.OutOfMemoryError: PermGen space
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at org.gradle.internal.classloader.MultiParentClassLoader.loadClass(MultiParentClassLoader.java:63)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
    at org.gradle.internal.classloader.CachingClassLoader.loadClass(CachingClassLoader.java:41)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
    at org.gradle.internal.classloader.FilteringClassLoader.loadClass(FilteringClassLoader.java:80)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
    at org.gradle.internal.classloader.CachingClassLoader.loadClass(CachingClassLoader.java:41)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at org.apache.tools.ant.AntClassLoader.findBaseClass(AntClassLoader.java:1385)
    at org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1080)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at ch.qos.logback.classic.spi.PackagingDataCalculator.loadClass(PackagingDataCalculator.java:202)
    at ch.qos.logback.classic.spi.PackagingDataCalculator.bestEffortLoadClass(PackagingDataCalculator.java:221)
    at ch.qos.logback.classic.spi.PackagingDataCalculator.computeBySTEP(PackagingDataCalculator.java:136)
    at ch.qos.logback.classic.spi.PackagingDataCalculator.populateUncommonFrames(PackagingDataCalculator.java:111)
    at ch.qos.logback.classic.spi.PackagingDataCalculator.populateFrames(PackagingDataCalculator.java:103)
    at ch.qos.logback.classic.spi.PackagingDataCalculator.calculate(PackagingDataCalculator.java:55)

Pastebin of messages.log: http://pastebin.com/uD8KnpaX Pastebin of threaddump: http://pastebin.com/ipXwH7Vc

Upvotes: 0

Views: 399

Answers (1)

Ryan Walls
Ryan Walls

Reputation: 7092

Added this line to gradle.properties

org.gradle.jvmargs=-XX:+UseConcMarkSweepGC -Xmx2048m -Xms512m -server -XX:MaxPermSize=512M

And it seems to have resolved the issue.

My prior attempts failed because I was updating GRADLE_OPTS which apparently does not pass memory changes to the spawned tomcat instance.

Upvotes: 1

Related Questions