Oracle 23ai RAC 설치

귀찮아서 미뤄 두었던 23ai RAC 를설치해 보았다. Single 설치에 이은 2탄이라고나 할까? 

단, Exadata 용 설치 binary 이기 때문에 운영 용도로 성급히 사용해서는 절대 안된다. 반드시 반드시 테스트 용도로만 사용해 볼것!!!

 

1. Oracle Grid Infrastructure S/W 설치

- Directory 사전 생성
mkdir -p /u01/app/23.0.0.0/grid
chown -R oracle:dba /u01

1. 설치파일을 FTP로 해당서버에 upload
LINUX.X64_235000_forEngineeredSytems_grid_home.zip (Size 2.8G)
 
2. 설치이미지 압축해제 (반드시 $GI_HOME 밑에 압축해제 해야 함)
unzip LINUX.X64_235000_forEngineeredSytems_grid_home.zip -d $GI_HOME
 
3. gridSetup.sh실행

xhost +
su – oracle
export DISPLAY=작업PC IP:0.0
cd $GI_HOME
./ gridSetup.sh 실행
 

[Configure Oracle Grid Infrastructure for a New Cluster 선택후, Next]

[Configure Clusterto manage Oracle RAC databases, Next]

[Cluster Name, SCAN Name 지정후, Next]

[add 버튼으로 rac2노드 추가 후 passwordless ssh , Next]

[Network Interface 지정 후, Next]

[Use Oracle Flex ASM for storage 선택 후, Next]

[VOTE 디스크 지정 후, Next] 

여기서는 HIGH로 구성함

[비밀번호 지정 후, Next]

[Enable Automatic Self Correction uncheck, Next]

[Do not use IPMI , Next]

[EM 미구성, Next]

[dba 그룹지정후, Next]

[GI Base 지정 후, Next]

[Inventory 지정후, Next]

[자동 스크립트 uncheck후, Next]

[Fixup 후 Ignore All check 후, Next]

[rsp 파일 저장후, Install Click] 

[화면의 내용대로 반드시 노드별 순서대로 스크립트 실행]

orainstRoot.sh 수행 (모든 노드 수행)

 

1번노드에서 수행
root@rose1:/root# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
 
Changing groupname of /u01/app/oraInventory to dba.
The execution of the script is complete.

2번노드에서 수행
root@rose2:/root# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
 
Changing groupname of /u01/app/oraInventory to dba.
The execution of the script is complete.
 
1번노드에서 수행
root@rose1:/root# /u01/app/23.0.0.0/grid/root.sh
Performing root user operation.
 
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/23.0.0.0/grid
 
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
 
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
RAC option enabled on: Linux
Executing command '/u01/app/23.0.0.0/grid/perl/bin/perl -I/u01/app/23.0.0.0/grid/perl/lib -I/u01/app/23.0.0.0/grid/crs/install /u01/app/23.0.0.0/grid/crs/install/rootcrs.pl '
Using configuration parameter file: /u01/app/23.0.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/rose1/crsconfig/rootcrs_rose1_2024-08-07_03-50-49PM.log
2024/08/07 15:50:56 CLSRSC-594: Executing installation step 1 of 19: 'ValidateEnv'.
2024/08/07 15:50:56 CLSRSC-594: Executing installation step 2 of 19: 'CheckRootCert'.
2024/08/07 15:50:57 CLSRSC-594: Executing installation step 3 of 19: 'GenSiteGUIDs'.
2024/08/07 15:50:59 CLSRSC-594: Executing installation step 4 of 19: 'SetupOSD'.
2024/08/07 15:50:59 CLSRSC-594: Executing installation step 5 of 19: 'CheckCRSConfig'.
2024/08/07 15:51:00 CLSRSC-594: Executing installation step 6 of 19: 'SetupLocalGPNP'.
2024/08/07 15:51:01 CLSRSC-594: Executing installation step 7 of 19: 'CreateRootCert'.
2024/08/07 15:51:02 CLSRSC-594: Executing installation step 8 of 19: 'ConfigOLR'.
2024/08/07 15:51:02 CLSRSC-594: Executing installation step 9 of 19: 'ConfigCHMOS'.
2024/08/07 15:51:02 CLSRSC-594: Executing installation step 10 of 19: 'CreateOHASD'.
2024/08/07 15:51:03 CLSRSC-594: Executing installation step 11 of 19: 'ConfigOHASD'.
2024/08/07 15:51:13 CLSRSC-594: Executing installation step 12 of 19: 'SetupTFA'.
2024/08/07 15:51:13 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2024/08/07 15:51:13 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2024/08/07 15:51:16 CLSRSC-594: Executing installation step 15 of 19: 'CheckFirstNode'.
2024/08/07 15:51:17 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
2024/08/07 15:51:21 CLSRSC-4002: Successfully installed Oracle Autonomous Health Framework (AHF).
CRS-4256: Updating the profile
Successful addition of voting disk 70302270ae374f52bfdb52dce11423c6.
Successful addition of voting disk 853b00134e034fcebf88f25576fa2a80.
Successful addition of voting disk 7578baed74b14fd6bfcdf937816cc086.
Successful addition of voting disk def894b31fa94f55bf82a38e52a1eedf.
Successful addition of voting disk 31929cd8f29d4f0ebf1b91a21ee27fa6.
Successfully replaced voting disk group with +VOTE.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   70302270ae374f52bfdb52dce11423c6 (/dev/VOTE1) [VOTE]
 2. ONLINE   853b00134e034fcebf88f25576fa2a80 (/dev/VOTE2) [VOTE]
 3. ONLINE   7578baed74b14fd6bfcdf937816cc086 (/dev/VOTE3) [VOTE]
 4. ONLINE   def894b31fa94f55bf82a38e52a1eedf (/dev/VOTE4) [VOTE]
 5. ONLINE   31929cd8f29d4f0ebf1b91a21ee27fa6 (/dev/VOTE5) [VOTE]
Located 5 voting disk(s).
2024/08/07 15:52:29 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2024/08/07 15:52:57 CLSRSC-343: Successfully started Oracle Clusterware stack
2024/08/07 15:53:03 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
clscfg: EXISTING configuration version 23 detected.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
2024/08/07 15:53:41 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2024/08/07 15:54:13 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
 
2번노드에서 수행
root@rose2:/root#  /u01/app/23.0.0.0/grid/root.sh
Performing root user operation.
 
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/23.0.0.0/grid
 
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
 
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
RAC option enabled on: Linux
Executing command '/u01/app/23.0.0.0/grid/perl/bin/perl -I/u01/app/23.0.0.0/grid/perl/lib -I/u01/app/23.0.0.0/grid/crs/install /u01/app/23.0.0.0/grid/crs/install/rootcrs.pl '
Using configuration parameter file: /u01/app/23.0.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/rose2/crsconfig/rootcrs_rose2_2024-08-07_04-54-49PM.log
2024/08/07 16:54:56 CLSRSC-594: Executing installation step 1 of 19: 'ValidateEnv'.
2024/08/07 16:54:56 CLSRSC-594: Executing installation step 2 of 19: 'CheckRootCert'.
2024/08/07 16:54:57 CLSRSC-594: Executing installation step 3 of 19: 'GenSiteGUIDs'.
2024/08/07 16:54:57 CLSRSC-594: Executing installation step 4 of 19: 'SetupOSD'.
2024/08/07 16:54:57 CLSRSC-594: Executing installation step 5 of 19: 'CheckCRSConfig'.
2024/08/07 16:54:58 CLSRSC-594: Executing installation step 6 of 19: 'SetupLocalGPNP'.
2024/08/07 16:54:58 CLSRSC-594: Executing installation step 7 of 19: 'CreateRootCert'.
2024/08/07 16:54:58 CLSRSC-594: Executing installation step 8 of 19: 'ConfigOLR'.
2024/08/07 16:54:59 CLSRSC-594: Executing installation step 9 of 19: 'ConfigCHMOS'.
2024/08/07 16:54:59 CLSRSC-594: Executing installation step 10 of 19: 'CreateOHASD'.
2024/08/07 16:55:00 CLSRSC-594: Executing installation step 11 of 19: 'ConfigOHASD'.
2024/08/07 16:55:06 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2024/08/07 16:55:15 CLSRSC-594: Executing installation step 12 of 19: 'SetupTFA'.
2024/08/07 16:55:15 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2024/08/07 16:55:15 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2024/08/07 16:55:19 CLSRSC-594: Executing installation step 15 of 19: 'CheckFirstNode'.
2024/08/07 16:55:19 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
2024/08/07 16:55:21 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2024/08/07 16:56:15 CLSRSC-4002: Successfully installed Oracle Autonomous Health Framework (AHF).
2024/08/07 16:57:25 CLSRSC-343: Successfully started Oracle Clusterware stack
2024/08/07 16:57:25 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2024/08/07 16:57:25 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2024/08/07 16:57:43 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
 
 양 노드에서 root.sh가 완료되어 OK 하면 후속작업이 진행되고 완료된다.

[설치완료]
 

- crs 상태확인

root@rose1:/root# crsctl stat res -t

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details      

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.LISTENER.lsnr

               ONLINE  ONLINE       rose1                    STABLE

               ONLINE  ONLINE       rose2                    STABLE

ora.chad

               ONLINE  ONLINE       rose1                    STABLE

               ONLINE  ONLINE       rose2                    STABLE

ora.cvuadmin

               OFFLINE OFFLINE      rose1                    STABLE

               OFFLINE OFFLINE      rose2                    STABLE

ora.helper

               OFFLINE OFFLINE      rose1                    STABLE

               OFFLINE OFFLINE      rose2                    IDLE,STABLE

ora.net1.network

               ONLINE  ONLINE       rose1                    STABLE

               ONLINE  ONLINE       rose2                    STABLE

ora.ons

               ONLINE  ONLINE       rose1                    STABLE

               ONLINE  ONLINE       rose2                    STABLE

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)

      1        ONLINE  ONLINE       rose1                    STABLE

      2        ONLINE  ONLINE       rose2                    STABLE

ora.LISTENER_SCAN1.lsnr

      1        ONLINE  ONLINE       rose1                    STABLE

ora.VOTE.dg(ora.asmgroup)

      1        ONLINE  ONLINE       rose1                    STABLE

      2        OFFLINE OFFLINE                               STABLE

ora.asm(ora.asmgroup)

      1        ONLINE  ONLINE       rose1                    Started,STABLE

      2        ONLINE  ONLINE       rose2                    Started,STABLE

ora.asmnet1.asmnetwork(ora.asmgroup)

      1        ONLINE  ONLINE       rose1                    STABLE

      2        ONLINE  ONLINE       rose2                    STABLE

ora.cdp1.cdp

      1        ONLINE  ONLINE       rose1                    STABLE

ora.cvu

      1        ONLINE  ONLINE       rose1                    STABLE

ora.rhpserver

      1        OFFLINE OFFLINE                               STABLE

ora.rose1.vip

      1        ONLINE  ONLINE       rose1                    STABLE

ora.rose2.vip

      1        ONLINE  ONLINE       rose2                    STABLE

ora.scan1.vip

      1        ONLINE  ONLINE       rose1                    STABLE

--------------------------------------------------------------------------------

root@rose1:/root# crsctl stat res -t -init

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details      

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

      1        ONLINE  ONLINE       rose1                    Started,STABLE

ora.cluster_interconnect.haip

      1        ONLINE  ONLINE       rose1                    STABLE

ora.crf

      1        ONLINE  ONLINE       rose1                    STABLE

ora.crsd

      1        ONLINE  ONLINE       rose1                    STABLE

ora.cssd

      1        ONLINE  ONLINE       rose1                    STABLE

ora.cssdmonitor

      1        ONLINE  ONLINE       rose1                    STABLE

ora.diskmon

      1        OFFLINE OFFLINE                               STABLE

ora.evmd

      1        ONLINE  ONLINE       rose1                    STABLE

ora.gipcd

      1        ONLINE  ONLINE       rose1                    STABLE

ora.gpnpd

      1        ONLINE  ONLINE       rose1                    STABLE

ora.mdnsd

      1        ONLINE  ONLINE       rose1                    STABLE

ora.storage

      1        ONLINE  ONLINE       rose1                    STABLE

--------------------------------------------------------------------------------

 

2. ASMCA 이용 추가 Diskgroup 생성

- Diskgroup 생성전 asm_diskstring 추가작업
 
root@rose1:/root# su - oracle
[ROSE1]oracle@rose1:/home/oracle# . oraenv
ORACLE_SID = [ROSE1] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/23.0.0.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[+ASM1]oracle@rose1:/home/oracle# sa
 
SQL*Plus: Release 23.0.0.0.0 - for Oracle Cloud and Engineered Systems on Thu Aug 8 09:03:08 2024
Version 23.5.0.24.07
 Copyright (c) 1982, 2024, Oracle.  All rights reserved.
Connected to:
Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - for Oracle Cloud and Engineered Systems
Version 23.5.0.24.07
 
SQL> alter system set asm_diskstring='/dev/VOTE*','/dev/DATA*','/dev/RECO*';
System altered.
 
export DISPLAY=작업PC IP:0.0
 ./asmca

[왼쪽메뉴 Disk Groups 선택, Create Click]
[DATA 디스크 그룹지정, External 선택후 OK]
[RECO 디스크 그룹지정, External 선택후 OK]
[생성된 Diskgroup 확인후 Exit]

3. Oracle Database S/W 설치

- Directory 사전 생성
mkdir -p /u01/app/oracle/product/23.0.0.0/dbhome_1
chown -R oracle:dba /u01
chmod -R 755 /u01
 
1. 설치파일을 FTP로 해당서버에upload
LINUX.X64_235000_forEngineeredSystems_db_home.zip (Size 2.1G)
 
2. 설치이미지 압축해제 (반드시 $ORACLE_HOME 밑에 압축해제 해야 함)
LINUX.X64_235000_forEngineeredSystems_db_home.zip -d $ORACLE_HOME
 
3. runInstaller실행
xhost +
su – oracle
export DISPLAY=작업PC IP:0.0
cd $ORACLE_HOME
./runInstaller 실행
 

[Set Up RAC Software Only 지정 후, Next]

[Oralcle Real Application Clusters 선택후 Next]

[노드 지정후, Next]

[Enterprise Edition 지정 후, Next]

[Oracle base 지정 후, Next]

[그룹 지정 후, Next]

[Automatically run configuration scripts 선택하지 않는다, Next]

[Ignore all 체크후, Next]

[Summary 확인후, Install]

[설치 진행이 완료되면 화면의 script 를 순서대로 실행]

[root@rose1]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
 
Changing groupname of /u01/app/oraInventory to dba.
The execution of the script is complete.
 
[root@rose1]# /u01/app/oracle/product/23.0.0.0/dbhome_1/root.sh
Performing root user operation.
 
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME=  /u01/app/oracle/product/23.0.0.0/dbhome_1
 
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
 
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
 
2번도 동일하게 수행후 OK 

[Database S/W 설치완료]

4. Netca 이용 리스너 생성

GRID 설치시 구성되어 있으므로 생략 

 

5. DBCA 이용 Database 생성

export DISPLAY=작업PC IP:0.0
cd $ORACLE_HOME/bin
./dbca
 

[Create Database 선택 후, Next]

[Advanced configuration선택 후, Next]

[Custom Database 선택 후, Next]

[노드 지정 후, Next]

[db name/SID 지정, PDB 이름 지정 (252개까지 가능) 후, Next]

[Datafile 저장경로 설정 후, Next]

[Fast Recovery Area / 아카이브 경로 설정, Next]

[설치할 component 확인 후, Next]

[Character sets 확인 후, Next]

[EM Uncheck 후, Next]

[SYS, SYSTEM password 설정 후, Next]

[Generate database creation scripts Check후, Next]

[Prerequisite Check 진행, Ignore All 선택후, Next]

[Summary 확인후, Finish] 

[스크립트만 생성후, Close]
 

- Exadata version이기 때문에 engine설치는 잘 되지만 dbca생성시 아래와 같은 Error발 생합니다. (ORA-27350, ORA-01034) 

 

This version of the Oracle Database software cannot be run on this platform.

 

해결방법은 silent로 Database생성하여야 하며 아래와 같은 script 수정하여 dbca 돌리시면 됩니다.

우선 GUI dbca실행하여 script만 받아낸 후에…

[ROSEC1]oracle@rose1:/u01/app/oracle/admin/ROSEC/scripts# ls -al
total 97272
drwxr-x--- 2 oracle dba4096 Aug8 12:27 .
drwxr-x--- 7 oracle dba79 Aug8 10:42 ..
-rw-r----- 1 oracle dba849 Aug8 10:18 CreateClustDBViews.sql
-rw-r----- 1 oracle dba1948 Aug8 10:18 CreateDBCatalog.sql
-rw-r----- 1 oracle dba459 Aug8 10:18 CreateDBFiles.sql
-rw-r----- 1 oracle dba1201 Aug8 10:18 CreateDB.sql
-rw-r----- 1 oracle dba2512 Aug8 12:22 init.ora
-rw-r----- 1 oracle dba1758 Aug8 10:18 lockAccount.sql
-rw-r----- 1 oracle dba92 Aug8 10:18 mkDir.sql
-rw-r----- 1 oracle dba92 Aug8 10:18 PDBCreation.sql
-rw-r----- 1 oracle dba1591 Aug8 10:18 plug_ROSEP1.sql
-rw-r----- 1 oracle dba1266 Aug8 10:18 postDBCreation.sql
-rw-r----- 1 oracle dba1586 Aug8 10:18 postPDBCreation_ROSEP1.sql
-rwxr-x--- 1 oracle dba1131 Aug8 10:18 ROSEC1.sh
-rw-r----- 1 oracle dba      1314 Aug  8 10:18 ROSEC1.sql

 

vi init.ora

_exadata_feature_on=true   <--- 마지막 줄에 해당 hidden 추가후 저장

ROSEC1.sh실행하면 됨.

 

6. 설치완료후 확인

1번 노드
root@rose1:/root# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       rose1                    STABLE
               ONLINE  ONLINE       rose2                    STABLE
ora.chad
               ONLINE  ONLINE       rose1                    STABLE
               ONLINE  ONLINE       rose2                    STABLE
ora.cvuadmin
               OFFLINE OFFLINE      rose1                    STABLE
               OFFLINE OFFLINE      rose2                    STABLE
ora.helper
               OFFLINE OFFLINE      rose1                    IDLE,STABLE
               OFFLINE OFFLINE      rose2                    IDLE,STABLE
ora.net1.network
               ONLINE  ONLINE       rose1                    STABLE
               ONLINE  ONLINE       rose2                    STABLE
ora.ons
               ONLINE  ONLINE       rose1                    STABLE
               ONLINE  ONLINE       rose2                    STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.DATA.dg(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rose1                    STABLE
ora.RECO.dg(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.VOTE.dg(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    Started,STABLE
      2        ONLINE  ONLINE       rose2                    Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.cdp1.cdp
      1        ONLINE  ONLINE       rose1                    STABLE
ora.cvu
      1        ONLINE  ONLINE       rose1                    STABLE
ora.rhpserver
      1        OFFLINE OFFLINE                               STABLE
ora.rose1.vip
      1        ONLINE  ONLINE       rose1                    STABLE
ora.rose2.vip
      1        ONLINE  ONLINE       rose2                    STABLE
ora.rosec.db
      1        ONLINE  ONLINE       rose1                    Open,HOME=/u01/app/o
                                                             racle/product/23.0.0
                                                             .0/dbhome_1,STABLE
      2        ONLINE  ONLINE       rose2                    Open,HOME=/u01/app/o
                                                             racle/product/23.0.0
                                                             .0/dbhome_1,STABLE
ora.rosec.rosep1.pdb
      1        ONLINE  ONLINE       rose1                    READ WRITE,STABLE
      2        ONLINE  ONLINE       rose2                    READ WRITE,STABLE
ora.scan1.vip

      1        ONLINE  ONLINE       rose1                    STABLE

 
2번 노드
root@rose2:/root# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       rose1                    STABLE
               ONLINE  ONLINE       rose2                    STABLE
ora.chad
               ONLINE  ONLINE       rose1                    STABLE
               ONLINE  ONLINE       rose2                    STABLE
ora.cvuadmin
               OFFLINE OFFLINE      rose1                    STABLE
               OFFLINE OFFLINE      rose2                    STABLE
ora.helper
               OFFLINE OFFLINE      rose1                    IDLE,STABLE
               OFFLINE OFFLINE      rose2                    IDLE,STABLE
ora.net1.network
               ONLINE  ONLINE       rose1                    STABLE
               ONLINE  ONLINE       rose2                    STABLE
ora.ons
               ONLINE  ONLINE       rose1                    STABLE
               ONLINE  ONLINE       rose2                    STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.DATA.dg(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rose1                    STABLE
ora.RECO.dg(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.VOTE.dg(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    Started,STABLE
      2        ONLINE  ONLINE       rose2                    Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       rose1                    STABLE
      2        ONLINE  ONLINE       rose2                    STABLE
ora.cdp1.cdp
      1        ONLINE  ONLINE       rose1                    STABLE
ora.cvu
      1        ONLINE  ONLINE       rose1                    STABLE
ora.rhpserver
      1        OFFLINE OFFLINE                               STABLE
ora.rose1.vip
      1        ONLINE  ONLINE       rose1                    STABLE
ora.rose2.vip
      1        ONLINE  ONLINE       rose2                    STABLE
ora.rosec.db
      1        ONLINE  ONLINE       rose1                    Open,HOME=/u01/app/o
                                                             racle/product/23.0.0
                                                             .0/dbhome_1,STABLE
      2        ONLINE  ONLINE       rose2                    Open,HOME=/u01/app/o
                                                             racle/product/23.0.0
                                                             .0/dbhome_1,STABLE
ora.rosec.rosep1.pdb
      1        ONLINE  ONLINE       rose1                    READ WRITE,STABLE
      2        ONLINE  ONLINE       rose2                    READ WRITE,STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       rose1                    STABLE
--------------------------------------------------------------------------------