Rajeev
Rajeev

Reputation: 1196

ORA-12805: parallel query server died unexpectedly

We are getting below error frequently on one of our RAC instance. Because of the same error, the applicatin is getting hanged and causing problem.

Sat Aug 04 06:57:26 2012
IPC Send timeout detected.Sender: ospid 28201
Receiver: inst 1 binc 8 ospid 3537
GATHER_STATS_JOB encountered errors.  Check the trace file.
Errors in file /opt/oracle/db/diag/rdbms/mdspdb/mdspdb5/trace/mdspdb5_j001_28201.trc:
ORA-12805: parallel query server died unexpectedly

Again one more error which we are getting is as below:

Thu Aug 09 03:43:05 2012
IPC Send timeout detected.Sender: ospid 18645
Receiver: inst 1 binc 8 ospid 3537
GATHER_STATS_JOB encountered errors.  Check the trace file.
Errors in file /opt/oracle/db/diag/rdbms/mdspdb/mdspdb5/trace/mdspdb5_j002_18645.trc:
ORA-20011: Approximate NDV failed: ORA-12805: parallel query server died unexpectedly

The output from the trace file is:

Trace file /opt/oracle/db/diag/rdbms/mdspdb/mdspdb5/trace/mdspdb5_j001_22631.trc
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
ORACLE_HOME = /opt/oracle/db/product/11.1.0/db_1
System name:    Linux
Node name:  mdspdb05
Release:    2.6.16.46-0.12-smp
Version:    #1 SMP Thu May 17 14:00:09 UTC 2007
Machine:    x86_64
Instance name: mdspdb5
Redo thread mounted by this instance: 5
Oracle process number: 107
Unix process pid: 22631, image: oracle@mdspdb05 (J001)


*** 2012-08-15 03:43:12.742
*** SESSION ID:(486.34904) 2012-08-15 03:43:12.742
*** CLIENT ID:() 2012-08-15 03:43:12.742
*** SERVICE NAME:(SYS$USERS) 2012-08-15 03:43:12.742
*** MODULE NAME:(DBMS_SCHEDULER) 2012-08-15 03:43:12.742
*** ACTION NAME:(ORA$AT_OS_OPT_SY_40411) 2012-08-15 03:43:12.742

IPC Send timeout detected.Sender: ospid 22631
Receiver: inst 1 binc 8 ospid 3537
SKGXP: SKGXPCTX: 0x0x2abb52d24a38 ctx
SKGXP: 
SKGXP: WAIT HISTORY
SKGXP: Time(msec)    Wait Type   Return Code
SKGXP: ----------    ---------   ------------
SKGXP: 2501      NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 2500      NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 2503      NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 2501      NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 2503      NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 0         NORMAL      TIMEDOUT
SKGXP: 2500      NORMAL      TIMEDOUT
SKGXP: wait delta 61 sec (61423 msec) ctx ts 0x79e31f last ts 0x79e31f
SKGXP: user cpu time since last wait 0 sec 0 ticks
SKGXP: system cpu time since last wait 0 sec 0 ticks
SKGXP: locked 1
SKGXP: blocked 25048
SKGXP: timed wait receives 15
SKGXP: admno 0x66859bfb admport:
SKGXP: SSKGXPT 0x52d2597c flags  sockno 18 IP 192.168.1.10 UDP 61826
SKGXP: context timestamp 0x79e31f
SKGXP: buffers queued on port 0x2abb53449f50
SKGXP: 
SKGXP:    Dumping Connection Handle Table
SKGXP:     sconno     accono   ertt  state   seq#   RcvPid   TotCreditsSKGXP:   sent rtrans   acks
SKGXP:     CNH Table Bucket: 0
SKGXP: 0x03a8c901 0x50790ec9 46982647250960 46982647250948 46982647284070 46982647267591 46982647250952SKGXP:    363d      1d  33126d
SKGXP:     CNH Table Bucket: 1
SKGXP: 0x03a8c902 0x051d84d8     16 46982647250948 46982647284112 46982647259415 46982647250952SKGXP:    405d      0d  33168d
SKGXP:     CNH Table Bucket: 2
SKGXP: 0x03a8c903 0x51d441b2     16 46982647250948 46982647284217 46982647258187 46982647250952SKGXP:    510d      0d  33273d
SKGXP:     CNH Table Bucket: 3
SKGXP: 0x03a8c904 0x7d38c1af     16 46982647250948 46982647283952 46982647267615 46982647250952SKGXP:    245d      0d  33008d
SKGXP:     CNH Table Bucket: 4
SKGXP: 0x03a8c905 0x22782e02     16 46982647250948 46982647284097 46982647283080 46982647250952SKGXP:    390d      0d  33153d
SKGXP:     CNH Table Bucket: 5
SKGXP: 0x03a8c906 0x19e5dbdb     16 46982647250948 46982647284141 46982647283122 46982647250952SKGXP:    434d      0d  33197d
SKGXP:     CNH Table Bucket: 6
SKGXP: 0x03a8c907 0x024222c8     16 46982647250948 46982647283839 46982647258304 46982647250952SKGXP:    132d      0d  32895d
SKGXP:     CNH Table Bucket: 7
SKGXP: 0x03a8c908 0x1f3d5569     16 46982647250948 46982647284073 46982647283097 46982647250952SKGXP:    366d      0d  33129d
SKGXP:     CNH Table Bucket: 8
SKGXP: 0x03a8c909 0x4b4e08b3     16 46982647250948 46982647283990 46982647267605 46982647250952SKGXP:    283d      1d  33046d
SKGXP:     CNH Table Bucket: 9
SKGXP: 0x03a8c90a 0x32e47d79     16 46982647250948 46982647283856 46982647258242 46982647250952SKGXP:    149d      0d  32912d
SKGXP:     CNH Table Bucket: 10
SKGXP: 0x03a8c90b 0x60420275     16 46982647250948 46982647283829 46982647259471 46982647250952SKGXP:    122d      0d  32885d
SKGXP:     CNH Table Bucket: 11
SKGXP: 0x03a8c90c 0x5521db22     16 46982647250948 46982647283834 46982647259493 46982647250952SKGXP:    127d      0d  32890d
SKGXP:     CNH Table Bucket: 12
SKGXP: 0x03a8c90d 0x1696da78     16 46982647250948 46982647283708 46982647254481 46982647250952SKGXP:      1d    373d      0d
SKGXP: slot 3 rqh=0x2abb52d2e0e8 seq=32763 len=2144 accno=0x1696da78 start TS=0x2f31c rt TS=0x79e56c X CNT=374
SKGXP: 
SKGXP:    Dumping Accept Handle Table
SKGXP:        ach     accono     sconno      admno  state     SndPid   seq#    rcv rtrans   acks    credits
Dumping KSXP Connection Handle: cnh: 0x2abb54b97098
Link in Component Context : 0x2abb52fc0c08 <- cnh -> 0x2abb52fc0c08
Link in Connection Cache  : 0x2abb53070050 <- cnh -> 0x2abb52f68158
Pending Async Sends       : First: 0x2abb52d2e030 Last 0x2abb52d2e030
Name: 
State: 1 ==> OPEN
Flags: 0x0  ==> 
Connect request handle: (nil)
Miners Canary         : (nil)
OSD Connection handle : 0x2abb55286938
Stats pointer         : 0x19ee7f470
System incarnation when connection was made: 78
Dumping Target id for cnh
vers: 1 type: (1 ==> KSXP_TRANSPORT_ID_SKGXPPID) inst: 1 ptid: 401 ukey: 8
Protocol Info: vendor: 1 proto: 2 maj: 3 min: 0
pnum: 91 ospid: 3537
p1_pad1: 0 p2_pad1: (nil)
pad2: 0 pad3: 0
Dumping SKGXP Port ID: 0x2abb54b97120
SKGXP: SKGXPID 0x2abb54b97120 
SKGXP: skgxp port number 0x25a32042 process id 3537
SKGXP: admin port id
SKGXP: SKGXPGPID 0x7fff596feb58
SKGXP:  Internet address 192.168.1.12
SKGXP:  UDP port number 28958
Dumping connection stats 0x19ee7f470
Dumping SKGXP Connection Handle: osd cnh: 0x2abb55286938
SKGXP: SKGXPCNH: 0x2abb55286938 SKGXPCON_OPEN (2) sconno 3a8c90d accono 1696da78 admno fd8255e
SKGXP:  Remote admin port
SKGXP: SSKGXPT 0x55286974 flags SSKGXPT_WRITE sockno 18 IP 192.168.1.12 UDP 28958
SKGXP:  Remote data port
SKGXP: SSKGXPT 0x55286a08 flags SSKGXPT_WRITE sockno 18 IP 192.168.1.12 UDP 29092
SKGXP:  no disconnect notification request handle
SKGXP:   next seqno 32764 credits 7 ertt 16 resends on con 488510

*** 2012-08-15 03:43:12.792
Submitting synchronized dump request [268435460]

*** 2012-08-15 03:43:14.796
kxfp_send_callback
        Send timed out to slave 0 inst 1 (qref 0x19e82bf88)
kxfpgsg 
        Error 12805 encountered

*** 2012-08-15 03:43:14.831
ORA-20011: Approximate NDV failed: ORA-12805: parallel query server died unexpectedly

*** 2012-08-15 03:43:14.831
GATHER_STATS_JOB: GATHER_TABLE_STATS('"IMUSE01"','"SUBSCRIBEINFO"','""', ...)
ORA-20011: Approximate NDV failed: ORA-12805: parallel query server died unexpectedly

Oracle Version: Oracle 11.1.0.6.0 RAC

Upvotes: 0

Views: 13093

Answers (1)

Rajeev
Rajeev

Reputation: 1196

This was a bug in Oracle RAC. We upgraded the version to Oracle 11.1.0.7

The problem is solved now

Upvotes: 1

Related Questions