CODEWITHSUNDEEP
javajakarta-eetomcat
user3378974
user3378974

Reputation: 159

Tomcat crashes unexpectedly

In my java application tomcat shutdown unexpectedly.It only shows to the website that, 'The server is temporarily unable to service your request due to maintenance downtime or capacity problems.

Please try again later.
'Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8l DAV/2 mod_jk/1.2.31 Server at example.com Port 8080'

Also i have checked the catalina log and nothing much i found out. I am checking the err_pid file for jvm error, which says there was an error, but i cannot figure it out, i am not a server architect.

server details:

os: Red Hat Linux
jdk: 1.5.0_03
tomcat: 5.5.9
ram: 16 gb Here is the err_pid.log file. I think you will be able to find out the issue.

err_pid.log

# An unexpected error has been detected by HotSpot Virtual Machine:
#  SIGSEGV (0xb) at pc=0x00bfa6f6, pid=28643, tid=1309596576
# Java VM: Java HotSpot(TM) Server VM (1.5.0_03-b07 mixed mode)
# Problematic frame:
# C  [libc.so.6+0x276f6]  _setjmp+0x6
---------------  T H R E A D  ---------------

Current thread (0x4feb6748):  JavaThread "TP-Processor21" daemon [_thread_in_native, id=28707]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000010

Registers:
EAX=0x00000000, EBX=0x4dde35a8, ECX=0x08a1f558, EDX=0x00000010
ESP=0x4e0ec1b4, EBP=0x4e0ec270, ESI=0x5338a7e8, EDI=0x4feb6748
EIP=0x00bfa6f6, CR2=0x00000010, EFLAGS=0x00010246

Top of Stack: (sp=0x4e0ec1b4)
0x4e0ec1b4:   4dd8726e 00000010 4e0ec29c 00000001
0x4e0ec1c4:   080e6a38 4e0ec2a8 4e0ec238 4feb7298
0x4e0ec1d4:   080e6a24 4e0ec1f8 00000029 00000000
0x4e0ec1e4:   51335398 51c00068 00010000 00000001
0x4e0ec1f4:   00000001 51c00048 4e0ec278 4feb7298
0x4e0ec204:   00d00840 00000030 4e0ec278 00cf3f03
0x4e0ec214:   00cf3eb1 00000001 51c00048 00000000
0x4e0ec224:   4dd8fdf1 00000000 00cf5ab4 00000170 

Instructions: (pc=0x00bfa6f6)
0x00bfa6e6:   d1 59 5a c3 90 90 90 90 90 90 31 c0 8b 54 24 04
0x00bfa6f6:   89 5a 00 89 72 04 89 7a 08 8d 4c 24 04 89 4a 10 

Stack: [0x4e06d000,0x4e0ee000),  sp=0x4e0ec1b4,  free space=508k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libc.so.6+0x276f6]  _setjmp+0x6
C  [libfontmanager.so+0x45d4d]
C  [libfontmanager.so+0x46349]  Java_sun_font_FileFont_getGlyphImage+0xcb
j  sun.font.FileFont.getGlyphImage(JI)J+0
j  sun.font.FileFontStrike.getGlyphImagePtr(I)J+62
j  sun.font.FileFontStrike.getGlyphMetrics(I)Ljava/awt/geom/Point2D$Float;+25
j  sun.font.CompositeStrike.getGlyphMetrics(I)Ljava/awt/geom/Point2D$Float;+11
j  sun.font.StandardGlyphVector$GlyphStrike.addDefaultGlyphAdvance(ILjava/awt/geom/Point2D$Float;)V+5
j  sun.font.StandardGlyphVector.initPositions()V+106
j  sun.font.StandardGlyphVector.getLogicalBounds()Ljava/awt/geom/Rectangle2D;+5
j  org.krysalis.barcode4j.output.java2d.Java2DCanvasProvider.deviceText(Ljava/lang/String;DDDLjava/lang/String;DLorg/krysalis/barcode4j/TextAlignment;)V+39
j  org.krysalis.barcode4j.output.bitmap.BitmapCanvasProvider.deviceText(Ljava/lang/String;DDDLjava/lang/String;DLorg/krysalis/barcode4j/TextAlignment;)V+16
j  org.krysalis.barcode4j.output.Canvas.drawText(Ljava/lang/String;DDDLjava/lang/String;DLorg/krysalis/barcode4j/TextAlignment;)V+16
j  org.krysalis.barcode4j.impl.DrawingUtil.drawText(Lorg/krysalis/barcode4j/output/Canvas;Lorg/krysalis/barcode4j/impl/AbstractBarcodeBean;Ljava/lang/String;DDDLorg/krysalis/barcode4j/TextAlignment;)V+29
j  org.krysalis.barcode4j.impl.DefaultCanvasLogicHandler.endBarcode()V+168
j  org.krysalis.barcode4j.impl.code128.Code128LogicImpl.generateBarcodeLogic(Lorg/krysalis/barcode4j/ClassicBarcodeLogicHandler;Ljava/lang/String;)V+92
j  org.krysalis.barcode4j.impl.code128.Code128Bean.generateBarcode(Lorg/krysalis/barcode4j/output/CanvasProvider;Ljava/lang/String;)V+51
j  repaircenter.common.BarCode.render(Ljavax/servlet/ServletContext;Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+130
j  com.ge.appl.tsf.shells.web.TSFWebPresentationManager.callTSFPO(Lcom/ge/appl/tsf/shells/common/TSFParams;Ljava/lang/String;)V+46
j  com.ge.appl.tsf.shells.web.TSFWebPresentationManager.render(Lcom/ge/appl/tsf/shells/common/TSFParams;Lcom/ge/appl/tsf/kernel/TSFResponse;)V+202
j  com.ge.appl.tsf.shells.web.TSFDispatcher.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+728
j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+30
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+354
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  repaircenter.common.listingObjects.SessionFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+157
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+117
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+607
j  org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+285
j  org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+64
j  org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
j  org.apache.catalina.valves.FastCommonAccessLogValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  sun.font.FileFont.getGlyphImage(JI)J+0
j  sun.font.FileFontStrike.getGlyphImagePtr(I)J+62
j  sun.font.FileFontStrike.getGlyphMetrics(I)Ljava/awt/geom/Point2D$Float;+25
j  sun.font.CompositeStrike.getGlyphMetrics(I)Ljava/awt/geom/Point2D$Float;+11
j  sun.font.StandardGlyphVector$GlyphStrike.addDefaultGlyphAdvance(ILjava/awt/geom/Point2D$Float;)V+5
j  sun.font.StandardGlyphVector.initPositions()V+106
j  sun.font.StandardGlyphVector.getLogicalBounds()Ljava/awt/geom/Rectangle2D;+5
j  org.krysalis.barcode4j.output.java2d.Java2DCanvasProvider.deviceText(Ljava/lang/String;DDDLjava/lang/String;DLorg/krysalis/barcode4j/TextAlignment;)V+39
j  org.krysalis.barcode4j.output.bitmap.BitmapCanvasProvider.deviceText(Ljava/lang/String;DDDLjava/lang/String;DLorg/krysalis/barcode4j/TextAlignment;)V+16
j  org.krysalis.barcode4j.output.Canvas.drawText(Ljava/lang/String;DDDLjava/lang/String;DLorg/krysalis/barcode4j/TextAlignment;)V+16
j  org.krysalis.barcode4j.impl.DrawingUtil.drawText(Lorg/krysalis/barcode4j/output/Canvas;Lorg/krysalis/barcode4j/impl/AbstractBarcodeBean;Ljava/lang/String;DDDLorg/krysalis/barcode4j/TextAlignment;)V+29
j  org.krysalis.barcode4j.impl.DefaultCanvasLogicHandler.endBarcode()V+168
j  org.krysalis.barcode4j.impl.code128.Code128LogicImpl.generateBarcodeLogic(Lorg/krysalis/barcode4j/ClassicBarcodeLogicHandler;Ljava/lang/String;)V+92
j  org.krysalis.barcode4j.impl.code128.Code128Bean.generateBarcode(Lorg/krysalis/barcode4j/output/CanvasProvider;Ljava/lang/String;)V+51
j  repaircenter.common.BarCode.render(Ljavax/servlet/ServletContext;Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+130
j  com.ge.appl.tsf.shells.web.TSFWebPresentationManager.callTSFPO(Lcom/ge/appl/tsf/shells/common/TSFParams;Ljava/lang/String;)V+46
j  com.ge.appl.tsf.shells.web.TSFWebPresentationManager.render(Lcom/ge/appl/tsf/shells/common/TSFParams;Lcom/ge/appl/tsf/kernel/TSFResponse;)V+202
j  com.ge.appl.tsf.shells.web.TSFDispatcher.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+728
j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+30
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+354
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  repaircenter.common.listingObjects.SessionFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+157
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+117
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+607
j  org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+285
j  org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+64
j  org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
j  org.apache.catalina.valves.FastCommonAccessLogValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
v  ~C2IAdapter
J  org.apache.catalina.valves.AccessLogValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V
v  ~I2CAdapter
j  org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+42
j  org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+141
j  org.apache.jk.server.JkCoyoteHandler.invoke(Lorg/apache/jk/core/Msg;Lorg/apache/jk/core/MsgContext;)I+177
j  org.apache.jk.common.HandlerRequest.invoke(Lorg/apache/jk/core/Msg;Lorg/apache/jk/core/MsgContext;)I+330
j  org.apache.jk.common.ChannelSocket.invoke(Lorg/apache/jk/core/Msg;Lorg/apache/jk/core/MsgContext;)I+191
j  org.apache.jk.common.ChannelSocket.processConnection(Lorg/apache/jk/core/MsgContext;)V+101
j  org.apache.jk.common.SocketConnection.runIt([Ljava/lang/Object;)V+8
j  org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()V+167
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x080bc5e0 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=29000]
  0x4fefa568 JavaThread "Thread-54" daemon [_thread_blocked, id=28815]
  0x4febabb8 JavaThread "TP-Monitor" daemon [_thread_blocked, id=28712]
  0x4feb9e08 JavaThread "TP-Processor25" daemon [_thread_in_native, id=28711]
  0x4feb9058 JavaThread "TP-Processor24" daemon [_thread_blocked, id=28710]
  0x4feb82a8 JavaThread "TP-Processor23" daemon [_thread_blocked, id=28709]
  0x4feb74f8 JavaThread "TP-Processor22" daemon [_thread_blocked, id=28708]
=>0x4feb6748 JavaThread "TP-Processor21" daemon [_thread_in_native, id=28707]
  0x4feb59e8 JavaThread "TP-Processor20" daemon [_thread_blocked, id=28706]
  0x50bef758 JavaThread "TP-Processor19" daemon [_thread_blocked, id=28705]
  0x50bee9a8 JavaThread "TP-Processor18" daemon [_thread_blocked, id=28704]
  0x50bedbf8 JavaThread "TP-Processor17" daemon [_thread_blocked, id=28703]
  0x50bece48 JavaThread "TP-Processor16" daemon [_thread_blocked, id=28702]
  0x50bec098 JavaThread "TP-Processor15" daemon [_thread_in_Java, id=28701]
  0x50beb348 JavaThread "TP-Processor14" daemon [_thread_blocked, id=28700]
  0x50023110 JavaThread "TP-Processor13" daemon [_thread_blocked, id=28699]
  0x50022398 JavaThread "TP-Processor12" daemon [_thread_blocked, id=28698]
  0x500215e8 JavaThread "TP-Processor11" daemon [_thread_blocked, id=28697]
  0x50020838 JavaThread "TP-Processor10" daemon [_thread_blocked, id=28696]
  0x5001fa88 JavaThread "TP-Processor9" daemon [_thread_blocked, id=28695]
  0x5001ed10 JavaThread "TP-Processor8" daemon [_thread_blocked, id=28694]
  0x5001dfd8 JavaThread "TP-Processor7" daemon [_thread_blocked, id=28693]
  0x50bd7e10 JavaThread "TP-Processor6" daemon [_thread_blocked, id=28692]
  0x50bd7110 JavaThread "TP-Processor5" daemon [_thread_blocked, id=28691]
  0x50015be0 JavaThread "TP-Processor4" daemon [_thread_blocked, id=28690]
  0x50015760 JavaThread "TP-Processor3" daemon [_thread_blocked, id=28689]
  0x500152e0 JavaThread "TP-Processor2" daemon [_thread_blocked, id=28688]
  0x50014e60 JavaThread "TP-Processor1" daemon [_thread_blocked, id=28687]
  0x5073c1e0 JavaThread "http-159.82.103.149-19332-Monitor" [_thread_blocked, id=28686]
  0x5073b430 JavaThread "http-159.82.103.149-19332-Processor25" daemon [_thread_in_native, id=28685]
  0x5073a680 JavaThread "http-159.82.103.149-19332-Processor24" daemon [_thread_blocked, id=28684]
  0x507398d0 JavaThread "http-159.82.103.149-19332-Processor23" daemon [_thread_blocked, id=28683]
  0x50738b38 JavaThread "http-159.82.103.149-19332-Processor22" daemon [_thread_blocked, id=28682]
  0x5096aa88 JavaThread "http-159.82.103.149-19332-Processor21" daemon [_thread_blocked, id=28681]
  0x50969cd8 JavaThread "http-159.82.103.149-19332-Processor20" daemon [_thread_blocked, id=28680]
  0x50968f28 JavaThread "http-159.82.103.149-19332-Processor19" daemon [_thread_blocked, id=28679]
  0x50968178 JavaThread "http-159.82.103.149-19332-Processor18" daemon [_thread_blocked, id=28678]
  0x509673c8 JavaThread "http-159.82.103.149-19332-Processor17" daemon [_thread_blocked, id=28677]
  0x50a850d0 JavaThread "http-159.82.103.149-19332-Processor16" daemon [_thread_blocked, id=28676]
  0x50a84320 JavaThread "http-159.82.103.149-19332-Processor15" daemon [_thread_blocked, id=28675]
  0x50a835a8 JavaThread "http-159.82.103.149-19332-Processor14" daemon [_thread_blocked, id=28674]
  0x50a827f8 JavaThread "http-159.82.103.149-19332-Processor13" daemon [_thread_blocked, id=28673]
  0x50a81a78 JavaThread "http-159.82.103.149-19332-Processor12" daemon [_thread_blocked, id=28672]
  0x5015a8a0 JavaThread "http-159.82.103.149-19332-Processor11" daemon [_thread_blocked, id=28671]
  0x50159af0 JavaThread "http-159.82.103.149-19332-Processor10" daemon [_thread_blocked, id=28670]
  0x50158d40 JavaThread "http-159.82.103.149-19332-Processor9" daemon [_thread_blocked, id=28669]
  0x50157fc8 JavaThread "http-159.82.103.149-19332-Processor8" daemon [_thread_blocked, id=28668]
  0x50bb9c40 JavaThread "http-159.82.103.149-19332-Processor7" daemon [_thread_blocked, id=28667]
  0x50bb8f20 JavaThread "http-159.82.103.149-19332-Processor6" daemon [_thread_blocked, id=28666]
  0x50926180 JavaThread "http-159.82.103.149-19332-Processor5" daemon [_thread_blocked, id=28665]
  0x4fefbc20 JavaThread "http-159.82.103.149-19332-Processor4" daemon [_thread_blocked, id=28664]
  0x501fdbb0 JavaThread "http-159.82.103.149-19332-Processor3" daemon [_thread_blocked, id=28663]
  0x50a58ba8 JavaThread "http-159.82.103.149-19332-Processor2" daemon [_thread_blocked, id=28662]
  0x50a17d98 JavaThread "http-159.82.103.149-19332-Processor1" daemon [_thread_blocked, id=28661]
  0x083ea958 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=28660]
  0x51c4a9b0 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=28657]
  0x51c49590 JavaThread "CompilerThread1" daemon [_thread_in_native, id=28656]
  0x51c485d0 JavaThread "CompilerThread0" daemon [_thread_in_native, id=28655]
  0x51c475d0 JavaThread "AdapterThread" daemon [_thread_blocked, id=28654]
  0x51c46738 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=28653]
  0x51c3cc98 JavaThread "Finalizer" daemon [_thread_blocked, id=28652]
  0x51c3bf60 JavaThread "Reference Handler" daemon [_thread_blocked, id=28651]
  0x0805c0e8 JavaThread "main" [_thread_in_native, id=28643]

Other Threads:
  0x51c393f0 VMThread [id=28650]
  0x51c4bf50 WatcherThread [id=28658]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 PSYoungGen      total 51328K, used 18119K [0xab4b0000, 0xaee90000, 0xb2670000)
  eden space 43520K, 26% used [0xab4b0000,0xabfe4470,0xadf30000)
  from space 7808K, 85% used [0xae6f0000,0xaed6d820,0xaee90000)
  to   space 7872K, 0% used [0xadf30000,0xadf30000,0xae6e0000)
 PSOldGen        total 466048K, used 0K [0x72670000, 0x8ed90000, 0xab4b0000)
  object space 466048K, 0% used [0x72670000,0x72670000,0x8ed90000)
 PSPermGen       total 262144K, used 14441K [0x52670000, 0x62670000, 0x72670000)
  object space 262144K, 5% used [0x52670000,0x5348a508,0x62670000)

Also i got one error reported in catalina.log. Please have a look into that too..

    java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364)
        at java.net.Socket.connect(Socket.java:507)
        at java.net.Socket.connect(Socket.java:457)
        at java.net.Socket.<init>(Socket.java:365

)
    at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:394)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:320)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:411)

Upvotes: 1

Views: 5771

Answers (1)

Stephen C
Stephen C

Reputation: 718816

In the absence of any tangible evidence, I can offer a couple of observations that might help.

  1. You are using a really old version of Java. Java 5 was released in 2004, and 1.5.0_03 is a very early patch release. Public patches ceased in 2009.

  2. Hard JVM crashes (that result in an "err_pid" dump file) are typically attributable to bugs in 1st / 3rd native code libraries, and occasionally to JVM bugs.

  3. It is often possible to get clues as to what was happening by looking at the "err_pid" file.

My advice would be to upgrade your Java release, at least to the last publicly available Java 5 release, and ideally all the way to Java 7. A Tomcat upgrade is probably warranted ... and also a general blitz on applying security patches to the machine.

If you have the wherewithal, you should try all of these remedies out on your test / UAT system before trying it in production. Java upgrades and Tomcat upgrades have the potential to introduce problems, and you don't want experience them first in on a production server.

Secondly, if this is a once off event, the most productive thing you could do would be to save all of the evidence (log files, dump files, configurations, etc) and put the problem on one side. Now YMMV, but in general it is only worth investigating this kind of problem if it recurs. (For a start, with a once-off event, it is really difficult to know that you have identified the cause and implemented the correct remedy ...)

Upvotes: 3

Related Questions