pat98's always & forever

- Dataguard stop 절차

standby에서

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;

Primary에서

SQL> ALTER SYSTEM SET log_archive_dest_state_2='DEFER' SCOPE=BOTH SID='*';

        srvctl stop database -d RAC or shutdown immediate

- Dataguard start 절차

standby에서

SQL> startup mount

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DICONNECT;

primary에서

SQL> ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH SID='*';

        srvctl start database -d RAC  or startup

Exadata ExaWatcher 기본명령어

위치 /opt/oracle.ExaWatcher/

1. ExaWatcher 중지

# ./ExaWatcher.sh --stop

2. ExaWatcher 중지

# /opt/oracle.cellos/vldrun -script oswatcher

3. ExaWatcher 동작확인

# ps -ef | grep -i ExaWatcher

ExaWatcher 가 수집하는 정보

- Diskinfo.ExaWatcher

# Collection Module:    DiskinfoExaWatcher

# Collection Command:   /bin/cat /proc/diskstats

- IBCardInfo.ExaWatcher

# Collection Module:    IBCardInfoExaWatcher

# Collection Command:   /opt/oracle.ExaWatcher/IBCardInfoExaWatcher.sh 2>/dev/null

- IBprocs.ExaWatcher

# Collection Module:    IBprocsExaWatcher

# Collection Command:   /bin/cat /sys/class/net/bondib0/bonding/active_slave

# Collection Command:   /bin/cat /sys/class/net/ib0/carrier

# Collection Command:   /bin/cat /sys/class/net/ib1/carrier

# Collection Command:   /usr/sbin/ibstatus

# Collection Command:   /usr/bin/ibv_devinfo

- Iostat.ExaWatcher

# Collection Module:    IostatExaWatcher

# Collection Command:   /usr/bin/iostat -t -x  5  720

- LGWR.ExaWatcher

# Collection Module:    LGWRExaWatcher

# Collection Command:   /opt/oracle.ExaWatcher/LGWRExaWatcher.sh

- Lsof.ExaWatcher

# Collection Module:    LsofExaWatcher

# Collection Command:   /usr/sbin/lsof +c0 -w +L -b -R -i

# Collection Command:   /usr/sbin/lsof +c0 -w +L -b -R -U

# Collection Command:   /usr/sbin/lsof +c0 -w +L1 -b -R

- MegaRaidFW.ExaWatcher

# Collection Module:    MegaRaidFWExaWatcher

# Collection Command:   /opt/MegaRAID/MegaCli/MegaCli64 fwtermlog dsply -a0

- Meminfo.ExaWatcher

# Collection Module:    MeminfoExaWatcher

# Collection Command:   /bin/cat /proc/meminfo

- Mpstat.ExaWatcher

# Collection Module:    MpstatExaWatcher

# Collection Command:   /usr/bin/mpstat -P ALL  5  720

- Netstat.ExaWatcher

# Collection Module:    NetstatExaWatcher

# Collection Command:   /bin/netstat -a -i -n

# Collection Command:   /bin/netstat -s

# Collection Command:   /bin/netstat -n -p -l

- Ps.ExaWatcher

# Collection Module:    PsExaWatcher

# Collection Command:   /bin/ps -eo flags,s,ruser,pid,ppid,c,psr,pri,ni,addr,sz,wchan,stime,tty,time,cmd

- RDSinfo.ExaWatcher

# Collection Module:    RDSinfoExaWatcher

# Collection Command:   /opt/oracle.ExaWatcher/RDSinfoExaWatcher.sh 2>/dev/null

- Slabinfo.ExaWatcher

# Collection Module:    SlabinfoExaWatcher

# Collection Command:   /bin/cat /proc/slabinfo

- Top.ExaWatcher

# Collection Module:    TopExaWatcher

# Collection Command:   /usr/bin/top -b -d 5 -n 720

- Vmstat.ExaWatcher

# Collection Module:    VmstatExaWatcher

# Collection Command:   /usr/bin/vmstat  5  2 

고객이 생각하는 용량과 실제 사용할수 있는 용량이 차이가 난다. 왜일까?

- 엑사데이타 용량 계산법

Quarter Rack 의 경우
- 2 DB nodes
- 3 Storage Cells

 3 Cell 서버 (High Performance) 는 아래와 같이 계산될수 있다.

Step 1: RAW 사이즈 계산 

우선 전체 용량을 구해보면 아래와 같다.

전체 Raw 용량 => 각 Cell 당 12EA disk * 각 Disk 당 600G * 3 Cell

                 => 21,600GB 또는  21.09TB

그런데 각 Cell 서버의 1번,2번 디스크에서 OS Linux 영역으로 30G씩 할당이 된다. 그래서 기본으로 디스크 그룹은 남은 10개의 디스크의 30G 용량으로 생성된다.

그리하여 DBFS 디스크그룹은 아래와 같이 용량을 차지 한다.

각 Cell에서 10EA 디스크 * 각 디스크에서 30 GB 씩  * 3 cell  =   DBFS => 900 GB

개별 디스크 600G-30G = 570G 가 사용가능하며 전체 디스크가 12EA * 3 cell  이므로

570 * 12 * 3 = 20,520 GB or  20.03TB 

Step 2: 미러링을 고려해 보자

 NORMAL Redundancy 로 설정하여 이중화 되었으므로 용량의 반만 사용할수 있다.

 20.03/2 = 10.01TB.

Step 3: fault tolerance를 고려한 여유공간 확보 

Disk Failure Coverage(DFC) 를 고려하면 Cell 에서 디스크하나가 나갔을 때를 대비한 용량은 600G 디스크 하나의 용량과 같다.

NOTE: that is a very important part. You need to plan for a Disk failure coverage or for complete cell node failure coverge. The ORACLE support document "Understanding ASM Capacity and Reservation of Free Space in Exadata (Doc ID 1551288.1)" clearly describes the different strategies and their pros and cos.

NOTE: After the application of BPS10 (11.2.0.4.10) the default is now the Disk Failure Coverage rather than the Cell Failure Coverage.


그래서 디스크 용량 하나가 또 빠진다.  10.01 TB - 0.57TB =>  9.4 TB

Step 4: Disk 그룹의 할당비율을 결정한다. 기본이 6;4 이므로 40% 가  recovery area를 위한 공간이 된다.

그래서 최종적으로 계산된 용량은 아래와 같다.

 RECO Diskgroup  => 3.76 TB
 DATA Diskgroup  => 5.64TB

출처 :

http://jehanzebdba.blogspot.kr/2014/09/exadata-storage-space-calculation.html

Exadata Cell Upgrade 하기 

11.2.3.3.0.131014.1 -> 11.2.3.3.1.140708

Cell Upgrade 는 DB node 에서 진행한다.

[EXA1]root@exa1:/root# dcli -g cell_group -l root /opt/oracle.cellos/ipconf -verify

gtceladm01: Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf

gtceladm01: Done. Configuration file /opt/oracle.cellos/cell.conf passed all verification checks

gtceladm02: Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf

gtceladm02: Done. Configuration file /opt/oracle.cellos/cell.conf passed all verification checks

gtceladm03: Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf

gtceladm03: Done. Configuration file /opt/oracle.cellos/cell.conf passed all verification checks

압축해제

[EXA1]root@exa1:/root# unzip p19166601_112331_Linux-x86-64.zip

Archive:  p19166601_112331_Linux-x86-64.zip

   creating: patch_11.2.3.3.1.140708/

  inflating: patch_11.2.3.3.1.140708/sundcs_36p_repository_2.1.3_4.pkg  

  inflating: patch_11.2.3.3.1.140708/upgradeIBSwitch.sh  

   creating: patch_11.2.3.3.1.140708/linux.db.rpms/

  inflating: patch_11.2.3.3.1.140708/linux.db.rpms/perl-XML-Parser-2.34-6.1.2.2.1.x86_64.rpm  

   creating: patch_11.2.3.3.1.140708/plugins/

  inflating: patch_11.2.3.3.1.140708/plugins/010-check_17854520.sh  

  inflating: patch_11.2.3.3.1.140708/plugins/000-check_dummy_bash  

  inflating: patch_11.2.3.3.1.140708/plugins/000-check_dummy_perl  

  inflating: patch_11.2.3.3.1.140708/11.2.3.3.1.140708.iso  

....................

[EXA1]root@exa1:/root# crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'exa1'

CRS-2673: Attempting to stop 'ora.crsd' on 'exa1'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'exa1'

CRS-2673: Attempting to stop 'ora.registry.acfs' on 'exa1'

CRS-2673: Attempting to stop 'ora.exa.db' on 'exa1'

CRS-2673: Attempting to stop 'dbfs_mount' on 'exa1'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'exa1'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'exa1'

CRS-2677: Stop of 'dbfs_mount' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.fsdb.db' on 'exa1'

CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.scan1.vip' on 'exa1'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.exa1.vip' on 'exa1'

CRS-2677: Stop of 'ora.registry.acfs' on 'exa1' succeeded

CRS-2677: Stop of 'ora.fsdb.db' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.DBFS.dg' on 'exa1'

CRS-2677: Stop of 'ora.exa.db' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.DATA.dg' on 'exa1'

CRS-2673: Attempting to stop 'ora.RECO.dg' on 'exa1'

CRS-2677: Stop of 'ora.exa1.vip' on 'exa1' succeeded

CRS-2672: Attempting to start 'ora.exa1.vip' on 'exa2'

CRS-2677: Stop of 'ora.RECO.dg' on 'exa1' succeeded

CRS-2677: Stop of 'ora.DATA.dg' on 'exa1' succeeded

CRS-2677: Stop of 'ora.scan1.vip' on 'exa1' succeeded

CRS-2672: Attempting to start 'ora.scan1.vip' on 'exa2'

CRS-2676: Start of 'ora.exa1.vip' on 'exa2' succeeded

CRS-2676: Start of 'ora.scan1.vip' on 'exa2' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'exa2'

CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'exa2' succeeded

CRS-2677: Stop of 'ora.DBFS.dg' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'exa1'

CRS-2677: Stop of 'ora.asm' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.ons' on 'exa1'

CRS-2677: Stop of 'ora.ons' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on 'exa1'

CRS-2677: Stop of 'ora.net1.network' on 'exa1' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'exa1' has completed

CRS-2677: Stop of 'ora.crsd' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.mdnsd' on 'exa1'

CRS-2673: Attempting to stop 'ora.ctssd' on 'exa1'

CRS-2673: Attempting to stop 'ora.evmd' on 'exa1'

CRS-2673: Attempting to stop 'ora.asm' on 'exa1'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'exa1'

CRS-2677: Stop of 'ora.ctssd' on 'exa1' succeeded

CRS-2677: Stop of 'ora.evmd' on 'exa1' succeeded

CRS-2677: Stop of 'ora.mdnsd' on 'exa1' succeeded

CRS-2677: Stop of 'ora.asm' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'exa1'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'exa1'

CRS-2677: Stop of 'ora.drivers.acfs' on 'exa1' succeeded

CRS-2677: Stop of 'ora.cssd' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.diskmon' on 'exa1'

CRS-2673: Attempting to stop 'ora.crf' on 'exa1'

CRS-2677: Stop of 'ora.crf' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'exa1'

CRS-2677: Stop of 'ora.diskmon' on 'exa1' succeeded

CRS-2677: Stop of 'ora.gipcd' on 'exa1' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'exa1'

CRS-2677: Stop of 'ora.gpnpd' on 'exa1' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'exa1' has completed

CRS-4133: Oracle High Availability Services has been stopped.

[EXA1]root@exa1:/root# dcli -g /root/cell_group -l root 'service celld stop'

gtceladm01: 

gtceladm01: Stopping the RS, CELLSRV, and MS services...

gtceladm01: The SHUTDOWN of services was successful.

gtceladm02: 

gtceladm02: Stopping the RS, CELLSRV, and MS services...

gtceladm02: The SHUTDOWN of services was successful.

gtceladm03: 

gtceladm03: Stopping the RS, CELLSRV, and MS services...

gtceladm03: The SHUTDOWN of services was successful.

[EXA1]root@exa1:/root# dcli -g /root/cell_group -l root 'service celld status'

gtceladm01: rsStatus:            stopped

gtceladm01: msStatus:            unknown

gtceladm01: cellsrvStatus:       unknown

gtceladm02: rsStatus:            stopped

gtceladm02: msStatus:            unknown

gtceladm02: cellsrvStatus:       unknown

gtceladm03: rsStatus:            stopped

gtceladm03: msStatus:            unknown

gtceladm03: cellsrvStatus:       unknown

[EXA1]root@exa1:/root# 

[EXA1]root@exa1:/root#  dcli -g /root/cell_group -l root 'sync;sync;sync'

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708#  ./patchmgr -cells /root/cell_group -reset_force

2015-01-20 13:36:41 +0900 :DONE: reset_force

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708#  ./patchmgr -cells /root/cell_group -cleanup

2015-01-20 13:37:54 +0900        :Working: DO: Cleanup ...

2015-01-20 13:37:55 +0900        :SUCCESS: DONE: Cleanup

-ignore_alerts 옵션 없이

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708#  ./patchmgr -cells /root/cell_group -patch_check_prereq

2015-01-20 13:39:03 +0900        :Working: DO: Initialize files, check space and state of cell services. Up to 1 minute ...

FAILED for following cells

gtceladm01:  gtceladm01 172.16.40.101 2015-01-20 13:39:38 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm01.log for details. Correct this before retrying the patch.

gtceladm02:  gtceladm02 172.16.40.102 2015-01-20 13:20:53 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm02.log for details. Correct this before retrying the patch.

gtceladm03:  gtceladm03 172.16.40.103 2015-01-20 13:21:40 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm03.log for details. Correct this before retrying the patch.

2015-01-20 13:39:36 +0900        :FAILED: Details in files <cell_name>.log /root/patch_11.2.3.3.1.140708/patchmgr.stdout, /root/patch_11.2.3.3.1.140708/patchmgr.stderr

2015-01-20 13:39:36 +0900        :FAILED: DONE: Initialize files, check space and state of cell services.

[ERROR] Patch prerequisite checks failed. Please run cleanup before retrying.

-ignore_alerts 옵션 주고

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708#  ./patchmgr -cells /root/cell_group -patch_check_prereq -ignore_alerts

2015-01-20 13:46:10 +0900        :Working: DO: Initialize files, check space and state of cell services. Up to 1 minute ...

2015-01-20 13:46:27 +0900        :SUCCESS: DONE: Initialize files, check space and state of cell services.

2015-01-20 13:46:27 +0900        :Working: DO: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...

2015-01-20 13:46:41 +0900 Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.

2015-01-20 13:46:42 +0900        :SUCCESS: DONE: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.

2015-01-20 13:46:42 +0900        :Working: DO: Check prerequisites on all cells. Up to 2 minutes ...

2015-01-20 13:47:28 +0900        :SUCCESS: DONE: Check prerequisites on all cells.

2015-01-20 13:47:28 +0900        :Working: DO: Execute plugin check for Patch Check Prereq ...

2015-01-20 13:47:28 +0900 :INFO: Patchmgr plugin start: Prereq check for exposure to bug 17854520 v1.1. Details in logfile /root/patch_11.2.3.3.1.140708/patchmgr.stdout.

2015-01-20 13:47:28 +0900 :INFO: This plugin checks dbhomes across all nodes with oracle-user ssh equivalence, but only for those known to the local system. dbhomes that exist only on remote nodes must be checked manually.

2015-01-20 13:47:28 +0900 :SUCCESS: No exposure to bug 17854520 with non-rolling patching

2015-01-20 13:47:28 +0900        :SUCCESS: DONE: Execute plugin check for Patch Check Prereq.

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708#  ./patchmgr -cells /root/cell_group -patch

NOTE Cells will reboot during the patch or rollback process.

NOTE For non-rolling patch or rollback, ensure all ASM instances using

NOTE the cells are shut down for the duration of the patch or rollback.

NOTE For rolling patch or rollback, ensure all ASM instances using

NOTE the cells are up for the duration of the patch or rollback.

WARNING Do not start more than one instance of patchmgr.

WARNING Do not interrupt the patchmgr session.

WARNING Do not alter state of ASM instances during patch or rollback.

WARNING Do not resize the screen. It may disturb the screen layout.

WARNING Do not reboot cells or alter cell services during patch or rollback.

WARNING Do not open log files in editor in write mode or try to alter them.

NOTE All time estimates are approximate. Timestamps on the left are real.

NOTE You may interrupt this patchmgr run in next 60 seconds with control-c.

2015-01-20 13:51:22 +0900        :Working: DO: Initialize files, check space and state of cell services. Up to 1 minute ...

FAILED for following cells

gtceladm01:  gtceladm01 172.16.40.101 2015-01-20 13:51:32 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm01.log for details. Correct this before retrying the patch.

gtceladm02:  gtceladm02 172.16.40.102 2015-01-20 13:32:46 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm02.log for details. Correct this before retrying the patch.

gtceladm03:  gtceladm03 172.16.40.103 2015-01-20 13:33:33 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm03.log for details. Correct this before retrying the patch.

2015-01-20 13:51:30 +0900        :FAILED: Details in files <cell_name>.log /root/patch_11.2.3.3.1.140708/patchmgr.stdout, /root/patch_11.2.3.3.1.140708/patchmgr.stderr

2015-01-20 13:51:30 +0900        :FAILED: DONE: Initialize files, check space and state of cell services.

[ERROR] Patch prerequisite checks failed. Please run cleanup before retrying.

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708# 

================PatchMgr run started 2015-01-20 13:50:15 +0900 ===========

With arguments: -cells /root/cell_group -patch

2015-01-20 13:51:22 +0900        :Working: DO: Initialize files, check space and state of cell services. Up to 1 minute ...

2015-01-20 13:51:29 +0900 ++++++++++++++++++ Logs so far begin ++++++++++

gtceladm01: 

gtceladm01: 2015-01-20 13:51:26 +0900 [INFO]: patchmgr launch attempt from exa1.gtplus.co.kr_172.16.20.1_root_patch_11.2.3.3.1.140708.

gtceladm01: 2015-01-20 13:51:26 +0900 [INFO]: dostep called: prechk:no:1486 patch_prereq -no-auto-rollback non_rolling 900 3600 900 36000 default noforce

gtceladm01: _EXIT_ERROR_Cell gtceladm01 172.16.40.101 2015-01-20 13:51:32 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm01.log for details. Correct this before retrying the patch.

gtceladm02: 

gtceladm02: 2015-01-20 13:32:41 +0900 [INFO]: patchmgr launch attempt from exa1.gtplus.co.kr_172.16.20.1_root_patch_11.2.3.3.1.140708.

gtceladm02: 2015-01-20 13:32:41 +0900 [INFO]: dostep called: prechk:no:1486 patch_prereq -no-auto-rollback non_rolling 900 3600 900 36000 default noforce

gtceladm02: _EXIT_ERROR_Cell gtceladm02 172.16.40.102 2015-01-20 13:32:46 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm02.log for details. Correct this before retrying the patch.

gtceladm03: 

gtceladm03: 2015-01-20 13:33:28 +0900 [INFO]: patchmgr launch attempt from exa1.gtplus.co.kr_172.16.20.1_root_patch_11.2.3.3.1.140708.

gtceladm03: 2015-01-20 13:33:28 +0900 [INFO]: dostep called: prechk:no:1486 patch_prereq -no-auto-rollback non_rolling 900 3600 900 36000 default noforce

gtceladm03: _EXIT_ERROR_Cell gtceladm03 172.16.40.103 2015-01-20 13:33:33 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm03.log for details. Correct this before retrying the patch.

gtceladm01: 

gtceladm01: _EXIT_PASS_Cell gtceladm01 172.16.40.101

gtceladm02: 

gtceladm02: _EXIT_PASS_Cell gtceladm02 172.16.40.102

gtceladm03: 

gtceladm03: _EXIT_PASS_Cell gtceladm03 172.16.40.103

gtceladm01: 

gtceladm01: [INFO] Free space in /boot (/dev/md4) before clean up is 81Mb

gtceladm01: [INFO] Kernel version on runtime: 2.6.39-400.126.1.el5uek

gtceladm01: [INFO] Kernel version in /opt/oracle.cellos/image.id: 2.6.39-400.126.1.el5uek

gtceladm01: [INFO] Free space in /boot (/dev/md4) after clean up is 81Mb

gtceladm01: [INFO] Size for all files for the kernel 2.6.39-400.126.1.el5uek in /boot is 23Mb

gtceladm01: [INFO] Required free space on /boot is 46Mb

gtceladm01: [ERROR] Detected open hardware alert(s) on the Exadata cell

gtceladm01: [ERROR] 3_1  2014-03-08T04:00:01+09:00       critical        "A power supply component is suspected of causing a fault.  Component Name : /SYS/PS0  Fault class    : fault.chassis.env.power.loss  Fault message  : http://www.sun.com/msg/SPX86-8000-55"

gtceladm01: [ERROR] Check for open hardware alerts on Exadata cell by running "cellcli -e list alerthistory where endTime=null and alertShortName=Hardware and alertType=stateful and severity=critical"

gtceladm01: [ERROR] Specific alerts can be dropped using "cellcli -e drop alerthistory X_Y" where X_Y is the alert ID

gtceladm01: Cell gtceladm01 172.16.40.101

gtceladm01: _EXIT_ERROR_Cell gtceladm01 172.16.40.101 2015-01-20 13:51:32 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm01.log for details. Correct this before retrying the patch.

gtceladm02: 

gtceladm02: [INFO] Free space in /boot (/dev/md4) before clean up is 81Mb

gtceladm02: [INFO] Kernel version on runtime: 2.6.39-400.126.1.el5uek

gtceladm02: [INFO] Kernel version in /opt/oracle.cellos/image.id: 2.6.39-400.126.1.el5uek

gtceladm02: [INFO] Free space in /boot (/dev/md4) after clean up is 81Mb

gtceladm02: [INFO] Size for all files for the kernel 2.6.39-400.126.1.el5uek in /boot is 23Mb

gtceladm02: [INFO] Required free space on /boot is 46Mb

gtceladm02: [ERROR] Detected open hardware alert(s) on the Exadata cell

gtceladm02: [ERROR] 3_1  2014-03-10T10:12:43+09:00       critical        "A power supply component is suspected of causing a fault.  Component Name : /SYS/PS0  Fault class    : fault.chassis.env.power.loss  Fault message  : http://www.sun.com/msg/SPX86-8000-55"

gtceladm02: [ERROR] Check for open hardware alerts on Exadata cell by running "cellcli -e list alerthistory where endTime=null and alertShortName=Hardware and alertType=stateful and severity=critical"

gtceladm02: [ERROR] Specific alerts can be dropped using "cellcli -e drop alerthistory X_Y" where X_Y is the alert ID

gtceladm02: Cell gtceladm02 172.16.40.102

gtceladm02: _EXIT_ERROR_Cell gtceladm02 172.16.40.102 2015-01-20 13:32:46 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm02.log for details. Correct this before retrying the patch.

gtceladm03: 

gtceladm03: [INFO] Free space in /boot (/dev/md4) before clean up is 81Mb

gtceladm03: [INFO] Kernel version on runtime: 2.6.39-400.126.1.el5uek

gtceladm03: [INFO] Kernel version in /opt/oracle.cellos/image.id: 2.6.39-400.126.1.el5uek

gtceladm03: [INFO] Free space in /boot (/dev/md4) after clean up is 81Mb

gtceladm03: [INFO] Size for all files for the kernel 2.6.39-400.126.1.el5uek in /boot is 23Mb

gtceladm03: [INFO] Required free space on /boot is 46Mb

gtceladm03: [ERROR] Detected open hardware alert(s) on the Exadata cell

gtceladm03: [ERROR] 3_1  2014-03-10T11:50:46+09:00       critical        "A power supply component is suspected of causing a fault.  Component Name : /SYS/PS0  Fault class    : fault.chassis.env.power.loss  Fault message  : http://www.sun.com/msg/SPX86-8000-55"

gtceladm03: [ERROR] Check for open hardware alerts on Exadata cell by running "cellcli -e list alerthistory where endTime=null and alertShortName=Hardware and alertType=stateful and severity=critical"

gtceladm03: [ERROR] Specific alerts can be dropped using "cellcli -e drop alerthistory X_Y" where X_Y is the alert ID

gtceladm03: Cell gtceladm03 172.16.40.103

gtceladm03: _EXIT_ERROR_Cell gtceladm03 172.16.40.103 2015-01-20 13:33:33 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm03.log for details. Correct this before retrying the patch.

2015-01-20 13:51:30 +0900 ++++++++++++++++++ Logs so far end ++++++++++

FAILED for following cells

gtceladm01:  gtceladm01 172.16.40.101 2015-01-20 13:51:32 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm01.log for details. Correct this before retrying the patch.

gtceladm02:  gtceladm02 172.16.40.102 2015-01-20 13:32:46 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm02.log for details. Correct this before retrying the patch.

gtceladm03:  gtceladm03 172.16.40.103 2015-01-20 13:33:33 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm03.log for details. Correct this before retrying the patch.

2015-01-20 13:51:30 +0900        :FAILED: Details in files <cell_name>.log /root/patch_11.2.3.3.1.140708/patchmgr.stdout, /root/patch_11.2.3.3.1.140708/patchmgr.stderr

2015-01-20 13:51:30 +0900        :FAILED: DONE: Initialize files, check space and state of cell services.

[ERROR] Patch prerequisite checks failed. Please run cleanup before retrying.

================PatchMgr run ended 2015-01-20 13:51:30 +0900 ===========

gtceladm01:  gtceladm01 172.16.40.101 2015-01-20 13:51:32 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm01.log for details. Correct this before retrying the patch.

gtceladm02:  gtceladm02 172.16.40.102 2015-01-20 13:32:46 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm02.log for details. Correct this before retrying the patch.

gtceladm03:  gtceladm03 172.16.40.103 2015-01-20 13:33:33 +0900: Detected open alerts in the alert history for the Exadata cell. Check gtceladm03.log for details. Correct this before retrying the patch.

2015-01-20 13:51:30 +0900        :FAILED: Details in files <cell_name>.log /root/patch_11.2.3.3.1.140708/patchmgr.stdout, /root/patch_11.2.3.3.1.140708/patchmgr.stderr

2015-01-20 13:51:30 +0900        :FAILED: DONE: Initialize files, check space and state of cell services.

[ERROR] Patch prerequisite checks failed. Please run cleanup before retrying.

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708# 

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708# 

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708# 

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708# ./patchmgr -cells /root/cell_group -patch -ignore_alerts

NOTE Cells will reboot during the patch or rollback process.

NOTE For non-rolling patch or rollback, ensure all ASM instances using

NOTE the cells are shut down for the duration of the patch or rollback.

NOTE For rolling patch or rollback, ensure all ASM instances using

NOTE the cells are up for the duration of the patch or rollback.

WARNING Do not start more than one instance of patchmgr.

WARNING Do not interrupt the patchmgr session.

WARNING Do not alter state of ASM instances during patch or rollback.

WARNING Do not resize the screen. It may disturb the screen layout.

WARNING Do not reboot cells or alter cell services during patch or rollback.

WARNING Do not open log files in editor in write mode or try to alter them.

NOTE All time estimates are approximate. Timestamps on the left are real.

NOTE You may interrupt this patchmgr run in next 60 seconds with control-c.

2015-01-20 13:55:14 +0900        :Working: DO: Initialize files, check space and state of cell services. Up to 1 minute ...

2015-01-20 13:55:31 +0900        :SUCCESS: DONE: Initialize files, check space and state of cell services.

2015-01-20 13:55:31 +0900        :Working: DO: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...

2015-01-20 13:55:44 +0900 Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.

2015-01-20 13:55:45 +0900        :SUCCESS: DONE: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.

2015-01-20 13:55:45 +0900        :Working: DO: Check prerequisites on all cells. Up to 2 minutes ...

2015-01-20 13:56:31 +0900        :SUCCESS: DONE: Check prerequisites on all cells.

2015-01-20 13:56:31 +0900        :Working: DO: Copy the patch to all cells. Up to 3 minutes ...

2015-01-20 13:57:13 +0900        :SUCCESS: DONE: Copy the patch to all cells.

2015-01-20 13:57:15 +0900        :Working: DO: Execute plugin check for Patch Check Prereq ...

2015-01-20 13:57:15 +0900 :INFO: Patchmgr plugin start: Prereq check for exposure to bug 17854520 v1.1. Details in logfile /root/patch_11.2.3.3.1.140708/patchmgr.stdout.

2015-01-20 13:57:15 +0900 :INFO: This plugin checks dbhomes across all nodes with oracle-user ssh equivalence, but only for those known to the local system. dbhomes that exist only on remote nodes must be checked manually.

2015-01-20 13:57:15 +0900 :SUCCESS: No exposure to bug 17854520 with non-rolling patching

2015-01-20 13:57:16 +0900        :SUCCESS: DONE: Execute plugin check for Patch Check Prereq.

2015-01-20 13:57:16 +0900 1 of 5 :Working: DO: Initiate patch on cells. Cells will remain up. Up to 5 minutes ...

2015-01-20 13:57:26 +0900 1 of 5 :SUCCESS: DONE: Initiate patch on cells.

2015-01-20 13:57:26 +0900 2 of 5 :Working: DO: Waiting to finish pre-reboot patch actions. Cells will remain up. Up to 45 minutes ...

2015-01-20 13:58:26 +0900 Wait for patch pre-reboot procedures

||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ Minutes left 045||||| Minutes left 045||||| Minutes left 045///// Minutes left 045----- Minutes left 045\\\\\ 

.....................................

\\\ Minutes left 083||||| Minutes left 083||||| Minutes left 083///// Minutes left 083----- Minutes left 083\\\\\ Minutes left 083||||| Minutes left 083||||| Minutes left 083///// Minutes left 083----- Minutes left 083\\\\\ Minutes left 083||||| Minutes left 083||||| Minutes left 083///// Minutes left 083----- Minutes left 083\\\\\ Minutes left 083||||| Minutes left 083                                          

2015-01-20 15:01:23 +0900 4 of 5 :SUCCESS: DONE: Wait for cells to reboot and come online.

2015-01-20 15:01:23 +0900 5 of 5 :Working: DO: Check the state of patch on cells. Up to 5 minutes ...

2015-01-20 15:02:04 +0900 5 of 5 :SUCCESS: DONE: Check the state of patch on cells.

2015-01-20 15:02:04 +0900        :Working: DO: Execute plugin check for Post Patch ...

2015-01-20 15:02:05 +0900        :SUCCESS: DONE: Execute plugin check for Post Patch.

================PatchMgr run started 2015-01-20 13:54:08 +0900 ===========

With arguments: -cells /root/cell_group -patch -ignore_alerts

2015-01-20 13:55:14 +0900        :Working: DO: Initialize files, check space and state of cell services. Up to 1 minute ...

2015-01-20 13:55:31 +0900 ++++++++++++++++++ Logs so far begin ++++++++++

gtceladm01: 

gtceladm01: 2015-01-20 13:55:19 +0900 [INFO]: patchmgr launch attempt from exa1.gtplus.co.kr_172.16.20.1_root_patch_11.2.3.3.1.140708.

gtceladm01: 2015-01-20 13:55:19 +0900 [INFO]: dostep called: prechk:yes:1486 patch_prereq -no-auto-rollback non_rolling 900 3600 900 36000 default noforce

gtceladm01: _EXIT_PASS_Cell gtceladm01 172.16.40.101 2015-01-20 13:55:33 +0900:

gtceladm02: 

gtceladm02: 2015-01-20 13:36:34 +0900 [INFO]: patchmgr launch attempt from exa1.gtplus.co.kr_172.16.20.1_root_patch_11.2.3.3.1.140708.

gtceladm02: 2015-01-20 13:36:34 +0900 [INFO]: dostep called: prechk:yes:1486 patch_prereq -no-auto-rollback non_rolling 900 3600 900 36000 default noforce

gtceladm02: _EXIT_PASS_Cell gtceladm02 172.16.40.102 2015-01-20 13:36:48 +0900:

gtceladm03: 

gtceladm03: 2015-01-20 13:37:21 +0900 [INFO]: patchmgr launch attempt from exa1.gtplus.co.kr_172.16.20.1_root_patch_11.2.3.3.1.140708.

gtceladm03: 2015-01-20 13:37:21 +0900 [INFO]: dostep called: prechk:yes:1486 patch_prereq -no-auto-rollback non_rolling 900 3600 900 36000 default noforce

gtceladm03: _EXIT_PASS_Cell gtceladm03 172.16.40.103 2015-01-20 13:37:35 +0900:

gtceladm01: 

gtceladm01: _EXIT_PASS_Cell gtceladm01 172.16.40.101

gtceladm02: 

gtceladm02: _EXIT_PASS_Cell gtceladm02 172.16.40.102

gtceladm03: 

gtceladm03: _EXIT_PASS_Cell gtceladm03 172.16.40.103

gtceladm01: 

gtceladm01: [INFO] Free space in /boot (/dev/md4) before clean up is 81Mb

gtceladm01: [INFO] Kernel version on runtime: 2.6.39-400.126.1.el5uek

gtceladm01: [INFO] Kernel version in /opt/oracle.cellos/image.id: 2.6.39-400.126.1.el5uek

gtceladm01: [INFO] Free space in /boot (/dev/md4) after clean up is 81Mb

gtceladm01: [INFO] Size for all files for the kernel 2.6.39-400.126.1.el5uek in /boot is 23Mb

gtceladm01: [INFO] Required free space on /boot is 46Mb

gtceladm01: [INFO][check_allowed_version] Target: 11.2.3.3.1.140708 Current: 11.2.3.3.0.131014.1 Patch or rollback: 1486 Rolling: non_rolling

gtceladm01: Cell gtceladm01 172.16.40.101

gtceladm01: _EXIT_PASS_Cell gtceladm01 172.16.40.101 2015-01-20 13:55:33 +0900:

gtceladm02: 

gtceladm02: [INFO] Free space in /boot (/dev/md4) before clean up is 81Mb

gtceladm02: [INFO] Kernel version on runtime: 2.6.39-400.126.1.el5uek

gtceladm02: [INFO] Kernel version in /opt/oracle.cellos/image.id: 2.6.39-400.126.1.el5uek

gtceladm02: [INFO] Free space in /boot (/dev/md4) after clean up is 81Mb

gtceladm02: [INFO] Size for all files for the kernel 2.6.39-400.126.1.el5uek in /boot is 23Mb

gtceladm02: [INFO] Required free space on /boot is 46Mb

gtceladm02: [INFO][check_allowed_version] Target: 11.2.3.3.1.140708 Current: 11.2.3.3.0.131014.1 Patch or rollback: 1486 Rolling: non_rolling

gtceladm02: Cell gtceladm02 172.16.40.102

gtceladm02: _EXIT_PASS_Cell gtceladm02 172.16.40.102 2015-01-20 13:36:48 +0900:

gtceladm03: 

gtceladm03: [INFO] Free space in /boot (/dev/md4) before clean up is 81Mb

gtceladm03: [INFO] Kernel version on runtime: 2.6.39-400.126.1.el5uek

gtceladm03: [INFO] Kernel version in /opt/oracle.cellos/image.id: 2.6.39-400.126.1.el5uek

gtceladm03: [INFO] Free space in /boot (/dev/md4) after clean up is 81Mb

gtceladm03: [INFO] Size for all files for the kernel 2.6.39-400.126.1.el5uek in /boot is 23Mb

gtceladm03: [INFO] Required free space on /boot is 46Mb

gtceladm03: [INFO][check_allowed_version] Target: 11.2.3.3.1.140708 Current: 11.2.3.3.0.131014.1 Patch or rollback: 1486 Rolling: non_rolling

gtceladm03: Cell gtceladm03 172.16.40.103

gtceladm03: _EXIT_PASS_Cell gtceladm03 172.16.40.103 2015-01-20 13:37:35 +0900:

2015-01-20 13:55:31 +0900 ++++++++++++++++++ Logs so far end ++++++++++

2015-01-20 13:55:31 +0900        :SUCCESS: DONE: Initialize files, check space and state of cell services.

2015-01-20 13:55:31 +0900        :Working: DO: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...

2015-01-20 13:55:45 +0900 ++++++++++++++++++ Logs so far begin ++++++++++

gtceladm01: 

gtceladm01: 2015-01-20 13:55:37 +0900 [INFO]: patchmgr launch attempt from exa1.gtplus.co.kr_172.16.20.1_root_patch_11.2.3.3.1.140708.

gtceladm01: 2015-01-20 13:55:37 +0900 [INFO]: dostep called: 4p patch_prereq -no-auto-rollback non_rolling 900 3600 900 36000 default

gtceladm01: _EXIT_PASS_Cell gtceladm01 172.16.40.101 2015-01-20 13:55:37 +0900:

gtceladm01: 

gtceladm01: 2015-01-20 13:55:48 +0900 [INFO]: patchmgr launch attempt from exa1.gtplus.co.kr_172.16.20.1_root_patch_11.2.3.3.1.140708.

gtceladm01: 2015-01-20 13:55:48 +0900 [INFO]: dostep called: 4p_md11_fix_pw patch_prereq -no-auto-rollback non_rolling 900 3600 900 36000 default

gtceladm01: _EXIT_PASS_Cell gtceladm01 172.16.40.101 2015-01-20 13:55:48 +0900:

...............................................

gtceladm03: 2015-01-20 13:39:21 +0900 [INFO] dostep_fix_12545272: LSI disk timeout: 360 for /sys/block/sdj

gtceladm03: 2015-01-20 13:39:21 +0900 [INFO] dostep_fix_12545272: LSI block device: /sys/block/sdk

gtceladm03: 2015-01-20 13:39:21 +0900 [INFO] dostep_fix_12545272: LSI timeout is 60s for /sys/block/sdk. Change it to 300s

gtceladm03: 2015-01-20 13:39:21 +0900 [INFO] dostep_fix_12545272: LSI disk timeout: 360 for /sys/block/sdk

gtceladm03: 2015-01-20 13:39:21 +0900 [INFO] dostep_fix_12545272: LSI block device: /sys/block/sdl

gtceladm03: 2015-01-20 13:39:21 +0900 [INFO] dostep_fix_12545272: LSI timeout is 60s for /sys/block/sdl. Change it to 300s

gtceladm03: 2015-01-20 13:39:21 +0900 [INFO] dostep_fix_12545272: LSI disk timeout: 360 for /sys/block/sdl

gtceladm03: 2015-01-20 13:39:25 +0900 [INFO] Execute: nohup ./install.sh -patch -execute -standby -no-auto-rollback -first-boot-usb -use-upgrade-usb  -ignore-celld-services -force -force -force

gtceladm03: 2015-01-20 13:39:30 +0900 [INFO] pid file: /root/_patch_hctap_/_p_/install.pid : 15227 0 install.sh

gtceladm03: 2015-01-20 13:39:30 +0900 [INFO] pid 15227 is running as "0 S     0 15227     1  2  80   0 -  3437 pipe_w ?          0:00 /bin/bash ./install.sh -patch -execute -standby -no-auto-rollback -first-boot-usb -use-upgrade-usb -ignore-celld-services -force -force -force"

gtceladm03: Cell gtceladm03 172.16.40.103

gtceladm03: _EXIT_PASS_Cell gtceladm03 172.16.40.103 2015-01-20 13:39:30 +0900:

2015-01-20 13:57:26 +0900 ++++++++++++++++++ Logs so far end ++++++++++

2015-01-20 13:57:26 +0900 1 of 5 :SUCCESS: DONE: Initiate patch on cells.

2015-01-20 13:57:26 +0900 2 of 5 :Working: DO: Waiting to finish pre-reboot patch actions. Cells will remain up. Up to 45 minutes ...

.......................................

gtceladm03: _EXIT_PASS_Cell gtceladm03 172.16.40.103 2015-01-20 14:44:08 +0900: Found: 11.2.3.3.1.140708 state success Expected: 11.2.3.3.1.140708 state success.

2015-01-20 15:02:04 +0900 ++++++++++++++++++ Logs so far end ++++++++++

2015-01-20 15:02:04 +0900 5 of 5 :SUCCESS: DONE: Check the state of patch on cells.

2015-01-20 15:02:04 +0900        :Working: DO: Execute plugin check for Post Patch ...

2015-01-20 15:02:04 +0900 ++++++++++++++++++ Logs so far begin ++++++++++

2015-01-20 15:02:05 +0900 ++++++++++++++++++ Logs so far end ++++++++++

2015-01-20 15:02:05 +0900        :SUCCESS: DONE: Execute plugin check for Post Patch.

================PatchMgr run ended 2015-01-20 15:02:05 +0900 ===========

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708# dcli -g /root/cell_group -l root imageinfo

gtceladm01: 

gtceladm01: Kernel version: 2.6.39-400.128.17.el5uek #1 SMP Tue May 27 13:20:24 PDT 2014 x86_64

gtceladm01: Cell version: OSS_11.2.3.3.1_LINUX.X64_140708

gtceladm01: Cell rpm version: cell-11.2.3.3.1_LINUX.X64_140708-1

gtceladm01: 

gtceladm01: Active image version: 11.2.3.3.1.140708

gtceladm01: Active image activated: 2015-01-20 15:00:37 +0900

gtceladm01: Active image status: success

gtceladm01: Active system partition on device: /dev/md6

gtceladm01: Active software partition on device: /dev/md8

gtceladm01: 

gtceladm01: In partition rollback: Impossible

gtceladm01: 

gtceladm01: Cell boot usb partition: /dev/sdm1

gtceladm01: Cell boot usb version: 11.2.3.3.1.140708

gtceladm01: 

gtceladm01: Inactive image version: 11.2.3.3.0.131014.1

gtceladm01: Inactive image activated: 2015-01-06 17:03:56 +0900

gtceladm01: Inactive image status: success

gtceladm01: Inactive system partition on device: /dev/md5

gtceladm01: Inactive software partition on device: /dev/md7

gtceladm01: 

gtceladm01: Boot area has rollback archive for the version: 11.2.3.3.0.131014.1

gtceladm01: Rollback to the inactive partitions: Possible

gtceladm02: 

gtceladm02: Kernel version: 2.6.39-400.128.17.el5uek #1 SMP Tue May 27 13:20:24 PDT 2014 x86_64

gtceladm02: Cell version: OSS_11.2.3.3.1_LINUX.X64_140708

gtceladm02: Cell rpm version: cell-11.2.3.3.1_LINUX.X64_140708-1

gtceladm02: 

gtceladm02: Active image version: 11.2.3.3.1.140708

gtceladm02: Active image activated: 2015-01-20 14:42:04 +0900

gtceladm02: Active image status: success

gtceladm02: Active system partition on device: /dev/md6

gtceladm02: Active software partition on device: /dev/md8

gtceladm02: 

gtceladm02: In partition rollback: Impossible

gtceladm02: 

gtceladm02: Cell boot usb partition: /dev/sdm1

gtceladm02: Cell boot usb version: 11.2.3.3.1.140708

gtceladm02: 

gtceladm02: Inactive image version: 11.2.3.3.0.131014.1

gtceladm02: Inactive image activated: 2015-01-06 17:53:13 +0900

gtceladm02: Inactive image status: success

gtceladm02: Inactive system partition on device: /dev/md5

gtceladm02: Inactive software partition on device: /dev/md7

gtceladm02: 

gtceladm02: Boot area has rollback archive for the version: 11.2.3.3.0.131014.1

gtceladm02: Rollback to the inactive partitions: Possible

gtceladm03: 

gtceladm03: Kernel version: 2.6.39-400.128.17.el5uek #1 SMP Tue May 27 13:20:24 PDT 2014 x86_64

gtceladm03: Cell version: OSS_11.2.3.3.1_LINUX.X64_140708

gtceladm03: Cell rpm version: cell-11.2.3.3.1_LINUX.X64_140708-1

gtceladm03: 

gtceladm03: Active image version: 11.2.3.3.1.140708

gtceladm03: Active image activated: 2015-01-20 14:42:37 +0900

gtceladm03: Active image status: success

gtceladm03: Active system partition on device: /dev/md6

gtceladm03: Active software partition on device: /dev/md8

gtceladm03: 

gtceladm03: In partition rollback: Impossible

gtceladm03: 

gtceladm03: Cell boot usb partition: /dev/sdm1

gtceladm03: Cell boot usb version: 11.2.3.3.1.140708

gtceladm03: 

gtceladm03: Inactive image version: 11.2.3.3.0.131014.1

gtceladm03: Inactive image activated: 2015-01-06 16:57:02 +0900

gtceladm03: Inactive image status: success

gtceladm03: Inactive system partition on device: /dev/md5

gtceladm03: Inactive software partition on device: /dev/md7

gtceladm03: 

gtceladm03: Boot area has rollback archive for the version: 11.2.3.3.0.131014.1

gtceladm03: Rollback to the inactive partitions: Possible

[EXA1]root@exa1:/root/patch_11.2.3.3.1.140708# dcli -g /root/cell_group -l root imagehistory

gtceladm01: Version                              : 11.2.3.3.0.131014.1

gtceladm01: Image activation date                : 2015-01-06 17:03:56 +0900

gtceladm01: Imaging mode                         : fresh

gtceladm01: Imaging status                       : success

gtceladm01: 

gtceladm01: Version                              : 11.2.3.3.1.140708

gtceladm01: Image activation date                : 2015-01-20 15:00:37 +0900

gtceladm01: Imaging mode                         : out of partition upgrade

gtceladm01: Imaging status                       : success

gtceladm01: 

gtceladm02: Version                              : 11.2.3.3.0.131014.1

gtceladm02: Image activation date                : 2015-01-06 17:53:13 +0900

gtceladm02: Imaging mode                         : fresh

gtceladm02: Imaging status                       : success

gtceladm02: 

gtceladm02: Version                              : 11.2.3.3.1.140708

gtceladm02: Image activation date                : 2015-01-20 14:42:04 +0900

gtceladm02: Imaging mode                         : out of partition upgrade

gtceladm02: Imaging status                       : success

gtceladm02: 

gtceladm03: Version                              : 11.2.3.3.0.131014.1

gtceladm03: Image activation date                : 2015-01-06 16:57:02 +0900

gtceladm03: Imaging mode                         : fresh

gtceladm03: Imaging status                       : success

gtceladm03: 

gtceladm03: Version                              : 11.2.3.3.1.140708

gtceladm03: Image activation date                : 2015-01-20 14:42:37 +0900

gtceladm03: Imaging mode                         : out of partition upgrade

gtceladm03: Imaging status                       : success

1. TNSNAMES.ORA 반영

EXA =

   (DESCRIPTION =

     (ADDRESS = (PROTOCOL = TCP)(HOST = 172.16.20.3)(PORT = 1521))

     (ADDRESS = (PROTOCOL = TCP)(HOST = 172.16.20.4)(PORT = 1521))

     (LOAD_BALANCE = yes)

     (CONNECT_DATA =

       (SERVER = DEDICATED)

       (SERVICE_NAME = EXA)

     )

   )

2. test.sh 작성

vi test.sh

#!/bin/sh

count=0

while [ $count -lt 5 ]                                           # 5번 수행하겠다.

do                                                               # loop 시작

    count=`expr $count + 1`                                   # 카운터 증가 

    sqlplus -S system/welcome1@EXA @test.sql 

done

3. test.sql 작성

col "Instance" format a20

col "Host" format a20

col "Service Name" format a20

select sys_context('userenv', 'instance_name') "Instance",

       sys_context('userenv', 'server_host') "Host",

       sys_context('userenv', 'service_name') "Service Name"

from dual

/

exit;

4. sh test.sh 실행

[EXA1]oracle@exa1:/home/oracle# sh loop.sh

Instance             Host                 Service Name

-------------------- -------------------- --------------------

EXA2                 exa2                 EXA

Instance             Host                 Service Name

-------------------- -------------------- --------------------

EXA1                 exa1                 EXA

Instance             Host                 Service Name

-------------------- -------------------- --------------------

EXA1                 exa1                 EXA

Instance             Host                 Service Name

-------------------- -------------------- --------------------

EXA2                 exa2                 EXA

Instance             Host                 Service Name

-------------------- -------------------- --------------------

EXA1                 exa1                 EXA

RAC cluster 에서 로드밸런스가 잘 되고 있음을 확인할수 있다.

Transparent Gateway for Microsoft SQL Server(TG4MSQL) 에서 data type conversion 정리

Data Type Conversion

The gateway converts Microsoft SQL Server data types to Oracle data types as follows:

설치된 RAC 의 cluster 이름을 알고 싶을때

[EXA1]oracle@exa1:/u01/app/11.2.0.3/grid/bin# cemutlo -n

cluster-clu1

증상 : ORA-12170, TNS-12535 발생하며 client 접속이 끊기는 현상이 발생할 경우

원인 : 방화벽의 최대 IDLE TIMEOUT 을 초과하여 long query 가 수행되거나 JDBC THIN CONNECTION POOL 을 사용할 경우 발생할수 있다.

조치방법 :

$ORACLE_HOME/network/admin/sqlnet.ora 화일에

SQLNET.EXPIRE_TIME=10

이렇게 설정하면 DCOD(Dead Connection Detection)을 사용하여 비활성화 10분이후 서버가 클라이언트에게 작은 10 바이트의 probe 패킷을 보냄.

의미: 10분마다 check 하겠다.

Exadata install.sh 사용법

cd /opt/oracle.SupportTools/onecommand/linux-x64

./install.sh -cf gtplus-gt.xml -l

1. Validate Configuration File

2. Setup Required Files

3. Create Users

4. Setup Cell Connectivity

5. Verify Infiniband

6. Calibrate Cells

7. Create Cell Disks

8. Create Grid Disks

9. Install Cluster Software

10. Initialize Cluster Software

11. Install Database Software

12. Relink Database with RDS

13. Create ASM Diskgroups

14. Create Databases

15. Apply Security Fixes

16. Install Exachk

17. Create Installation Summary

18. Resecure Machine

./install.sh -cf gtplus-gt.xml -l    (목록보기)

./install.sh -cf gtplus-gt.xml -s 1  (Step 지정)

./install.sh -cf gtplus-gt.xml -u 1  (Step undo)

./install.sh -cf gtplus-gt.xml -r 1-5  (Step 1-5 쭉 진행)