Strange behavior with different java application export option

Question

I have java server application wich uses many libs (netty, guava, etc). I always export this application as one single .jar. When I run application in Eclipse, I didn't have any problems. But if I start app in console (Windows, or Ubuntu, doesn't matter), I have strange problem: ALL connection processes via sockets last toooo long. For example, simple http connection via HttpAsync or others (rabbitmq connection, etc.) lasts 1-2 min. But after connection completed, data sends/receives fast. I can't figure what the problem. As mentioned before, I use Eclipse for development.

As you know, you can export project 3 dif ways (in Eclipse):

1. Extract required libraries into JAR.
2. Package required libraries into JAR.
3. Copy required libraries into sub folder next to JAR.

So, when I used 2 option, I had problem. When I switched to 3d option (all .jars in folder near main .jar), problem was solved.

Generally there are no big difference between 2 and 3 option (in 2 all .jars just inside one jar). I thought that it was cause of extra time needed to load new classes in execution time from the jars. But problem occurs not only at start, but for all new connections.

Can someone explain this behavior?

UPD: Eclipse Luna. Doesn't matter what OS I'm using (Windows, or Ubuntu), even doesn't matter what jvm (tried with different Oracle jdk, even tried open jdk).

Answer 1

This all talks about difference in performance when packaging into JAR v/s extracting into JAR & difference in performance when running from Eclipse v/s running from console.

Difference in performance when packaging into JAR v/s extracting into JAR:

Extract required libraries into JAR:

What it does:
In this option Eclipse will extract all the classes from the referenced JARs and package into the generated JAR.

If you open the JAR then you will find that there are NO referenced JARs packaged but all the classes of referenced JARs are arranged as per the package structure and then packaged inside the JAR at root level. This brings the key difference in performance as compared to the "Packaging required libraries into a jar file" where there is additionally cost of runtime parsing and loading of JAR in memory etc..

When exporting as JAR through Eclipse then it is best option if performance is concern. Also this is scalable option because you can ship this JAR

MANIFEST.MF Main thing to note in this file is you main class. When you run the JAR you are directly running the class you need.

Main-Class: com.my.jar.TestSSL

Package required libraries into JAR:

What it does:
In this option Eclipse will:

• package all the referenced JARs into the generated JAR.
• employ Eclipse's JAR loading mechanism through org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader and you can also see org.eclipse.jdt.internal.jarinjarloader package into your generated JAR and this package is just under the root directory of the generated JAR.

Now of course this is the additional cost which comes when you choose this option because when you run the JAR then it is not you main class getting executed but JarRsrcLoader will be executed which will load your main class and other libraries, and all the referenced libraries are packaged. See MANIFEST.MF section below

MANIFEST.MF Main thing to note in this file is you main class. When you run the JAR, JarRsrcLoader will run and will do further job.

Rsrc-Main-Class: com.cgi.tmi.TestSSL
Main-Class: org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader

Now for last Eclipse export option - "Copy required libraries into sub folder next to JAR", I don't think it is a very scalable solution to consider because this imposes your file system dependency, so I would say don't do it.

Difference in performance when running from Eclipse v/s running from console:

When you run application from Eclipse then it is quiet similar to 1st export option where Eclipse doesn't need to parse and load JARs at runtime and all.
This is however a very trivial point, key is the consideration of Eclipse JAR export option 1 v/s option 2.

---

Final words:

• Use "Extract required libraries into JAR" for exporting JAR and you will see substantial performance gain.
  • It is highly improbable that your socket connections are lasting long when you run from console because JVM runs code then it would have same or very comparable performance when running from Eclipse and console (considering same Java versions in both case). You could be feeling because of packaged JAR performance. Try extracted JAR and you should be fine.
• Also, consider the amount of logging you are doing. When running through, depending upon configuration Eclipse may mask a lot of logging and hence saving you i/o time.
• Do understand how classes are accessed from JAR class path, which is like additional computational cost when you are referencing classes from JAR.

Answer 2

It happens because when you go with "uber jar" approach, some metadata might be lost.

It's just an example, but if you download this and this, take a look inside the jar. There are a few files with the same name in the same META-INF folder.

Those files might be important, and when eclipse repackages things for you, he might not be the doing a decent job on merging such files.

That is what might be happening to you.

Answer 3

In the 2nd approach, You have all dependency jars in the main.jar. So it won't load any of the dependency jars unless required. Whereas, in case of 3rd option, your main.jar and other dependency jars are independent (unlike 2nd way), and hence gets loaded for connections and is available.

try adding a log statement or syso by manipulating a dependency jar to see this working.

Answer 4

As we don't know the exact structure of your JAR here is a more general explanation (assumed you run your application with java -jar your_app.jar).

case Copy required libraries into sub folder next to JAR.

• if a class needs to be loaded the class loader (after the runtime JAR) first checks your_app.jar to find a required class
• if the class is not found it traversed over all JAR files in the subfolder
• all JAR files could be kept in the filesystem cache for further reading

case Package required libraries into JAR

• if a class needs to be loaded the Eclipse class loader JarRsrcLoader (after the runtime JAR) first checks your_app.jar to find a required class
• if the class is not found it traversed over all embedded JAR files, which means as first they need to be decompressed from your_app.jar before the content can be read
• the extracted embedded JAR files are not kept in the filesystem cache for further reading (as they are not files in the filesystem)

If you have a bigger number of hugh embedded library JARs this might lead in a slow down of class loading (but only for the first time a class is loaded by a class loader).

You can see the difference in the class loading if you compare the outpout of

java -verbose:class -jar your_app_external_library_jars.jar

with

java -verbose:class -jar your_app_embedded_library_jars.jar

The performance might be improved by generating an INDEX.LIST file for each JAR file (e.g. your_app.jar and the embedded library JARs).