Oracle RAC patch 후 startup 안될때.

RAC 환경에서 patch 후 한쪽 노드가 올라오지 않을때.
alert.log 나 udump, cdump 등에도 특별한 로그는 남지 않음. 올리면 아래와 같이 나오기만 하고 난감함.

ORA-00469: CKPT process terminated with error
ORA-07445: exception encountered: core dump [] [] [] [] [] []

9i or 10g 상관없이 에러 발생함.

잘 찾아보면 1번 노드에는 $ORACLE_HOME/rdbms/lib/libknlopt.a 화일이 있으나 2번에는 없음.

수동으로 동일한 위치에 copy 해주고 (DB 내리고 할것) 다시 relink 해주면 한방에 올라옴.

요로코롬 해결하면 됨. 자세한 방법은 아래의 metalink 문서 참조.

==================================================================================================

Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 10.2.0.2
This problem can occur on any platform.
After installing any One-off patch in RAC nodes, one of those nodes may fail to start the instance.


NOTE: We did observe the similar problem on 10.2.0.2 RAC installations where certain libraries are not copied from the installing node to other RAC nodes. For such type of problem while performing installation, please refer to the following note.

Note 363840.1 - 10.2.0.2 RAC: ORA-7445 in KKXSYN or KKXCMS ON ALL NODES BUT INSTALL NODE

Symptoms

Oracle Instance does not startup on one of RAC the nodes after installing any One-off Patch.

If you see the alert and trace log files, you may see the following errors:

ORA-00469: CKPT process terminated with error

ORA-07445: exception encountered: core dump [] [] [] [] [] []

Cause

It was noted that the resulting 'Oracle' binaries on different nodes were not the same. Running a cksum on all involved libraries in lib/ and rdbms/lib/, it was evident that libknlopt.a was different across nodes. The object modules that get into libknlopt.a however are identical, so it clearly appears that libknlopt.a isn't relinked on nodes other than the installation one.

Solution

Copy the good library in the faulty node and manually re-link the Oracle again.

Ensure all instances are shut down !

Manually copy good $ORACLE_HOME/rdbms/lib/libknlopt.a from the local node to all remote faulty nodes and relink Oracle again on all faulty nodes.

eg: cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk ioracle