One of our two node Rac DB encounter 04031 due to inbalance in mini sub heap.
After turned parameter for shared_pool, i decided to bounce instance one by one.
But when i was bouncing the instance on first node, the second node instance also crashed.
Below is in the logfile for second instance:
Wed Jun 27 04:57:28 2012
Reconfiguration started (old inc 4, new inc 6)
List of instances:
2 (myinst: 2)
Global Resource Directory frozen
* dead instance detected - domain 0 invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Wed Jun 27 04:57:28 2012
LMS 1: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Wed Jun 27 04:57:28 2012
LMS 2: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Wed Jun 27 04:57:28 2012
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Wed Jun 27 04:57:28 2012
Instance recovery: looking for dead threads
Beginning instance recovery of 1 threads
parallel recovery started with 6 processes
Started redo scan
Completed redo scan
read 202 KB redo, 74 data blocks need recovery
Wed Jun 27 04:57:32 2012
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /d001/product/ora_base/diag/rdbms/ttpoltd/ttpoltd2/trace/ttpoltd2_lms2_942.trc:
ORA-04031: unable to allocate 4136 bytes of shared memory ("shared pool","unknown object","sga heap(3,0)","gcs dynamic r")
LMS2 (ospid: 942): terminating the instance due to error 4031
Wed Jun 27 04:57:33 2012
System state dump requested by (instance=2, osid=942 (LMS2)), summary=[abnormal instance termination].
System State dumped to trace file /d001/product/ora_base/diag/rdbms/ttpoltd/ttpoltd2/trace/ttpoltd2_diag_920.trc
Wed Jun 27 04:57:34 2012
ORA-1092 : opitsk aborting process
Errors in file /d001/product/ora_base/diag/rdbms/ttpoltd/ttpoltd2/trace/ttpoltd2_diag_920.trc:
ORA-00601: cleanup lock conflict
Dumping diagnostic data in directory=[cdmp_20120627045733], requested by (instance=2, osid=942 (LMS2)), summary=[abnormal instance termination].
Wed Jun 27 04:57:35 2012
ORA-1092 : opitsk aborting process
Wed Jun 27 04:57:35 2012
License high water mark = 38
Instance terminated by LMS2, pid = 942
USER (ospid: 17522): terminating the instance
Instance terminated by USER, pid = 17522
From above log we can see during instance reconfigure progress, LMS hit 04031 and failed to progress with reconfigure operation, hence LMS terminated the instance.:
0 Comments:
Post a Comment