RAC를 신규구성 완료후 Check 중 PDB 구조에서 UNDO Tablespace 가 이상하게 생성되어 있는 것이 확인 되었다.
그냥 써도 되지만 이름과 화일명이 왠지 불편하다..

PDB1         SYSAUX          /oradata/CDB/PDB1/sysaux01.dbf                    240,640               63,872       73.5%
PDB1         SYSTEM          /oradata/CDB/PDB1/system01.dbf                    307,200                4,992       98.4%
PDB1         UNDOTBS1     /oradata/CDB/PDB1/undotbs01.dbf                   250,880                5,120       98.0%
PDB1         UNDO_4         /oradata/CDB/PDB1/system01_i4_undo.dbf        97,664               94,848        2.9%
PDB1         USERS            /oradata/CDB/PDB1/users01.dbf                       7,168                  960       86.6%

local undo-enabled 가 설정된 RAC 환경에서 Node2 의 PDB 내의 UNDO tablespace 가 UNDO_2 로 생성되는 현상이다. (나의 경우에는 2node 임에도 UNDO_4 로 생성됨)
- 현재 product 기능으로는 정상적인 operation 이며 UNDOTBS2 라는 올바른 이름으로 자동생성될수 있게 Enhancement 가 요청된 상태라고 한다.
- 일종의 Bug 로 보이는데 Bug 로 인정하지는 않는 듯 하다.

환경 : 23.26.3.0.0

- 증상

1. undo local mode 를 확인한다.
column PROPERTY_NAME format a40
column PROPERTY_VALUE format a30
 SELECT PROPERTY_NAME, PROPERTY_VALUE FROM DATABASE_PROPERTIES WHERE PROPERTY_NAME = 'LOCAL_UNDO_ENABLED';

PROPERTY_NAME                            PROPERTY_VALUE
---------------------------------------- ------------------------------
LOCAL_UNDO_ENABLED                       TRUE

2. pdb 를 확인한다.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB1                           READ WRITE NO

3. 1번 노드에서 UNDO 확인
 
set linesize 200
column VALUE format a30
 select NAME,VALUE,CON_ID from V$SYSTEM_PARAMETER where NAME = 'undo_tablespace';

NAME                                                                             VALUE                              CON_ID
-------------------------------------------------------------------------------- ------------------------------ ----------
undo_tablespace                                                                  UNDOTBS1                              0
undo_tablespace                                                                                                                2
undo_tablespace                                                                  UNDO_4                                  3    <--- 생뚱맞게 UNDO_4 로 셋팅되어 있다.

4. 2번 노드에서 UNDO 확인
set linesize 200
column VALUE format a30
select NAME,VALUE,CON_ID from V$SYSTEM_PARAMETER where NAME = 'undo_tablespace';

NAME                                                                             VALUE                              CON_ID
-------------------------------------------------------------------------------- ------------------------------ ----------
undo_tablespace                                                                  UNDOTBS2                                0
undo_tablespace                                                                                                                  2
undo_tablespace                                                                  UNDOTBS1                                3   <--- 얘는 UNDOTBS2 이여야 하는데 UNDOTBS1 로 셋팅되어 있다.

- 정상화 작업

1. 현재 UNDO Tablespace 확인
SQL>  alter session set container=PDB1;
Session altered.

SQL> show parameter undo_tablespace

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
undo_tablespace                      string      UNDO_4


2. 새로운 이름으로 UNDO tablespace 를 생성해 준다.

SQL> create UNDO TABLESPACE UNDOTBS2 datafile '/oradata/CDB/PDB1/undotbs02.dbf' size 10M autoextend on;
Tablespace created.

생성이 잘 되었는지 확인
SQL> SELECT * FROM DBA_DATA_FILES;

FILE_NAME                                   FILE_ID TABLESPACE_NAME      BYTES     BLOCKS STATUS    RELATIVE_FNO AUT   MAXBYTES  MAXBLOCKS INCREMENT_BY USER_BYTES USER_BLOCKS ONLINE_ LOST_WR
---------------------------------------- ---------- --------------- ---------- ---------- --------- ------------ --- ---------- ---------- ------------ ---------- ----------- ------- -------
/oradata/CDB/PDB1/system01.dbf                    9 SYSTEM           314572800      38400 AVAILABLE         1024 YES 3.5184E+13 4294967293     1280  313524224       38272 SYSTEM  OFF
/oradata/CDB/PDB1/sysaux01.dbf                   10 SYSAUX           246415360      30080 AVAILABLE         1024 YES 3.5184E+13 4294967293     1280  240091136       29308 ONLINE  OFF
/oradata/CDB/PDB1/undotbs01.dbf                  11 UNDOTBS1         256901120      31360 AVAILABLE            6 YES 3.4360E+10    4194302      640  255852544       31232 ONLINE  OFF
/oradata/CDB/PDB1/system01_i4_undo.dbf           12 UNDO_4           100007936      12208 AVAILABLE           12 YES 3.4360E+10    4194302      640   98959360       12080 ONLINE  OFF
/oradata/CDB/PDB1/users01.dbf                    13 USERS              7340032        896 AVAILABLE         1024 YES 3.5184E+13 4294967293      160    1015808         124 ONLINE  OFF
/oradata/CDB/PDB1/undotbs02.dbf                  14 UNDOTBS2          10485760       1280 AVAILABLE         1024 YES 3.5184E+13 4294967293        1    9437184        1152 ONLINE  OFF

6 rows selected.

3. UNDO Tablespace 를 바꿔준다.

SQL> alter system set undo_tablespace=UNDOTBS1 container=current sid='CDB1' scope=spfile; (1번노드에서)
SQL> alter system set undo_tablespace=UNDOTBS2 container=current sid='CDB2' scope=spfile; (2번노드에서)

 

System altered.

4. 재기동  (1.2번 모두)

SQL> shutdown 
Pluggable Database closed.
SQL> startup
Pluggable Database opened.

5. UNDO_4 Segment 가 모두 사라졌는지 확인해 준다.

SQL> select dr.tablespace_name, dr.segment_name, vr.status from dba_rollback_segs dr, v$rollstat vr where dr.segment_id=vr.usn;

TABLESPACE_NAME SEGMENT_NAME                   STATUS
--------------- ------------------------------ ---------------
SYSTEM          SYSTEM                         ONLINE
UNDOTBS1        _SYSSMU1_1573790497$           ONLINE
UNDOTBS1        _SYSSMU2_971013227$            ONLINE
UNDOTBS1        _SYSSMU3_3502236436$           ONLINE
UNDOTBS1        _SYSSMU4_3322552541$           ONLINE
UNDOTBS1        _SYSSMU5_100379797$            ONLINE
UNDOTBS1        _SYSSMU6_211076933$            ONLINE
UNDOTBS1        _SYSSMU7_2708109441$           ONLINE
UNDOTBS1        _SYSSMU8_3680568534$           ONLINE
UNDOTBS1        _SYSSMU9_4115705703$           ONLINE
UNDOTBS1        _SYSSMU10_1726320278$          ONLINE

6. UNDO_4 Tablespace 를 삭제해 준다.

SQL> drop tablespace UNDO_4 including contents and datafiles;

확인
SQL> SELECT * FROM DBA_DATA_FILES;

FILE_NAME                                   FILE_ID TABLESPACE_NAME      BYTES     BLOCKS STATUS    RELATIVE_FNO AUT   MAXBYTES  MAXBLOCKS INCREMENT_BY USER_BYTES USER_BLOCKS ONLINE_ LOST_WR
---------------------------------------- ---------- --------------- ---------- ---------- --------- ------------ --- ---------- ---------- ------------ ---------- ----------- ------- -------
/oradata/CDB/PDB1/system01.dbf                    9 SYSTEM           314572800      38400 AVAILABLE         1024 YES 3.5184E+13 4294967293     1280  313524224       38272 SYSTEM  OFF
/oradata/CDB/PDB1/sysaux01.dbf                   10 SYSAUX           246415360      30080 AVAILABLE         1024 YES 3.5184E+13 4294967293     1280  240091136       29308 ONLINE  OFF
/oradata/CDB/PDB1/undotbs01.dbf                  11 UNDOTBS1         256901120      31360 AVAILABLE            6 YES 3.4360E+10    4194302      640  255852544       31232 ONLINE  OFF
/oradata/CDB/PDB1/users01.dbf                    13 USERS              7340032        896 AVAILABLE         1024 YES 3.5184E+13 4294967293      160    1015808         124 ONLINE  OFF
/oradata/CDB/PDB1/undotbs02.dbf                  14 UNDOTBS2          10485760       1280 AVAILABLE         1024 YES 3.5184E+13 4294967293        1    9437184        1152 ONLINE  OFF

 

깔끔해 졌다.

끝.

Posted by pat98

19c 까지는 되었는데 21c 이후부터 Clusterware 와의 연동이 desupport 되었다.

 

공유영역으로 구성이 가능한지 확인해 보기 위해 테스트 진행해 보았다. 

 

사실 공식적으로 지원하지 않는다고 명시된 부분이라 절대 현업에는 적용하면 안될 것이다.

 

https://docs.oracle.com/en/database/oracle/oracle-database/23/upgrd/oracle-database-changes-deprecations-desupports.html#GUID-3AAE48F4-68CC-469C-B682-CBE521B19AEF


Desupport of Vendor Clusterware Integration with Oracle Clusterware

Starting with Oracle Clusterware 21c, the integration of vendor or third party clusterware with Oracle Clusterware is desupported.

Date: May 2021

The integration of vendor clusterware with Oracle Clusterware is desupported in Oracle Database 21c. Desupporting certain clustering features with limited adoption allows Oracle to focus on improving core scaling, availability, and manageability across all features and functionality. In the absence of an integration between different cluster solutions, the system is subject to the dueling cluster solutions issue: Independent cluster solutions can make individual decisions about which corrective actions must be taken in case of certain failures. To avoid conflicts, only one cluster solution should be active at any point in time. For this reason, Oracle recommends that you align your next software or hardware upgrade with the transition off of vendor cluster solutions. This desupport applies to clustered deployments, including Oracle Real Application Clusters and Oracle RAC One Node.

 

OS : Oracle Linux 9.7

공유 클러스터 : OCFS2

오라클 GI 버전  : 23.26.3.0.0
39578862;MICRONAUT RELEASE UPDATE 23.26.3.0.0 (39578862) Gold Image
39578856;RHP RELEASE UPDATE 23.26.3.0.0 (39578856) Gold Image
39578865;ACFS RELEASE UPDATE 23.26.3.0.0 (39578865) Gold Image
39578859;OCW RELEASE UPDATE 23.26.3.0.0 (39578859) Gold Image
39578879;Database Release Update : 23.26.3.0.0 (39578879) Gold Image

 

우선 테스트를 위해 OCFS 로 아래와 같이 공유영역을 구성해 놓았다.

 

[root@ocfs1 ~]# df -h
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   4.0M     0  4.0M   0% /dev
tmpfs                      4.9G     0  4.9G   0% /dev/shm
tmpfs                      2.0G  8.9M  2.0G   1% /run
/dev/mapper/ol_ocfs1-root   11G  5.5G  5.5G  51% /
/dev/sda1                  960M  427M  534M  45% /boot
/dev/mapper/ol_ocfs1-u01    40G  318M   40G   1% /u01
/dev/sdb                    20G  1.1G   19G   6% /oradata
/dev/sdc                    10G  551M  9.5G   6% /archive
/dev/sdd                   2.0G  279M  1.8G  14% /vote01
/dev/sdf                   2.0G  279M  1.8G  14% /vote02
/dev/sde                   2.0G  279M  1.8G  14% /vote03
tmpfs                      995M   36K  995M   1% /run/user/0

 

 GI SW 를 구성하기 위해서는  VOTE 디스크가 공유영역에 있어야 하는데 아래와 같이 위치를 지정하면 26ai 은 19c 와 달리 이렇게 에러를 뿜어낸다.

 

 

Cause - The storage type of the location (/vote01/ocr01) is not supported for Oracle Cluster Registry.  
Action - Provide a supported storage location for the Oracle Cluster Registry.  
Additional Information:
- PRVG-11503 : Path "/vote01" of type "OCFS2" is not suitable for usage as Oracle Clusterware storage for release "23.0". Supported storage types are "NFS, ASM Disk Group, VXFS"
- Cause: The specified path is not found suitable for usage as Oracle Clusterware storage(OCR or Voting Disk)
- Action: Ensure that you have selected a path that is suitable for the desired usage. Summary of node specific errors ocfs2 ocfs1 
 

안내메세지는 위와 같이 클러스터 저장 영역으로는 NFS, ASM Disk, VXFS (베리타스 파일시스템) 3개만 지원한다. 

matrix 에도 나와 있지만 OCFS 는 원래 구성을 지원하지 않는다.

 

또한 infoscale 의 최신 matrix 에도 아직 26ai 는 지원되지 않는다. SFRAC 지원되지 않음.

 

ASM+CFS 혼용해서 같이 사용할 수도 있겠지만...

하지만 하지 말라고 하면 더 하고 싶어지는 법!!!

 

- ASM 으로 구성한 후에 OCFS 로 바꾸어서 공유영역으로 사용해 보자.


[root@ocfs1 ~]# df -h
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   4.0M     0  4.0M   0% /dev
tmpfs                      4.9G     0  4.9G   0% /dev/shm
tmpfs                      2.0G  8.9M  2.0G   1% /run
/dev/mapper/ol_ocfs1-root   11G  5.5G  5.5G  51% /
/dev/sda1                  960M  427M  534M  45% /boot
/dev/mapper/ol_ocfs1-u01    40G  318M   40G   1% /u01
/dev/sdb                    20G  1.1G   19G   6% /oradata
/dev/sdc                    10G  551M  9.5G   6% /archive
/dev/sdd                   2.0G  279M  1.8G  14% /vote01
/dev/sdf                   2.0G  279M  1.8G  14% /vote02
/dev/sde                   2.0G  279M  1.8G  14% /vote03
tmpfs                      995M   36K  995M   1% /run/user/0

현재는 공유영역으로 사용하기 위해 이렇게 구성이 되어 있다.

- 우선 디스크를 하나 더 붙이고 ASM 을 이용해서 VOTING FILE 이 저장될  디스크그룹을 만들어 준다.

만들어진 모습

[root@ocfs1]# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       ocfs1                    STABLE
               ONLINE  ONLINE       ocfs2                    STABLE
ora.chad
               ONLINE  ONLINE       ocfs1                    STABLE
               ONLINE  ONLINE       ocfs2                    STABLE
ora.helper
               OFFLINE OFFLINE      ocfs1                    ILDE,STABLE
               OFFLINE OFFLINE      ocfs2                    IDLE,STABLE
ora.net1.network
               ONLINE  ONLINE       ocfs1                    STABLE
               ONLINE  ONLINE       ocfs2                    STABLE
ora.ons
               ONLINE  ONLINE       ocfs1                    STABLE
               ONLINE  ONLINE       ocfs2                    STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       ocfs1                    STABLE
      2        ONLINE  ONLINE       ocfs2                    STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.VOTE.dg(ora.asmgroup)
      1        ONLINE  ONLINE       ocfs1                    STABLE
      2        ONLINE  ONLINE       ocfs2                    STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       ocfs1                    Started,STABLE
      2        ONLINE  ONLINE       ocfs2                    Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       ocfs1                    STABLE
      2        ONLINE  ONLINE       ocfs2                    STABLE
ora.cdp1.cdp
      1        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.cvuhelper
      1        OFFLINE OFFLINE                               STABLE
ora.ocfs1.vip
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.ocfs2.vip
      1        ONLINE  ONLINE       ocfs2                    STABLE
ora.rhpserver
      1        OFFLINE OFFLINE                               STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       ocfs1                    STABLE
--------------------------------------------------------------------------------

[root@ocfs1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   eada12b7b7504fdebfbb63aa9feb71d5 (/dev/sdg1) [VOTE]
Located 1 voting disk(s).

[root@ocfs1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          4
         Total space (kbytes)     :     901284
         Used space (kbytes)      :      84848
         Available space (kbytes) :     816436
         ID                       : 1972206939
         Device/File Name         :      +VOTE
                                    Device/File integrity check succeeded
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check succeeded
         Logical corruption check succeeded
 
- OCR과 VOTE 화일이 모두 +VOTE 디스크그룹에 저장되어 있다.

 

- 이제 공유영역으로 바꾸어 보자.
 
▣ VOTE 화일 이전

[root@ocfs1 ~]# crsctl replace votedisk /vote01/vote01
Now formatting voting disk: /vote01/vote01.
CRS-4256: Updating the profile
Successful addition of voting disk b73aa1ddef1c4f77bfcd2ecca87f789b.
Successful deletion of voting disk eada12b7b7504fdebfbb63aa9feb71d5.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced

[root@ocfs1 ~]# crsctl add css votedisk /vote02/vote02
Now formatting voting disk: /vote02/vote02.
CRS-4603: Successful addition of voting disk /vote02/vote02.

[root@ocfs1 ~]# crsctl add css votedisk /vote03/vote03
Now formatting voting disk: /vote03/vote03.
CRS-4603: Successful addition of voting disk /vote03/vote03.

[root@ocfs1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   b73aa1ddef1c4f77bfcd2ecca87f789b (/vote01/vote01) []
 2. ONLINE   6cffa04aaef44fc5bfe8689f37173c6c (/vote02/vote02) []
 3. ONLINE   a354031602514fdebf622bc1499ed207 (/vote03/vote03) []
Located 3 voting disk(s).

▣ OCR 화일 이전

[root@ocfs1 ~]# ocrconfig -add /vote01/ocr01
PROT-30: The Oracle Cluster Registry location to be added is not usable.
PROC-50: The Oracle Cluster Registry location to be added is inaccessible on nodes ocfs1, ocfs2.

화일이 없어서 에러가 난다. 빈 화일을 만들어 준다.

touch /vote01/ocr01 /vote01/ocr01 /vote01/ocr01
chown oracle:dba /vote01/ocr01 /vote02/ocr02 /vote03/ocr03
chmod 640 /vote01/ocr01 /vote02/ocr02 /vote03/ocr03 (voting file 인 경우는 chmod 644)



[root@ocfs1 vote01]# ocrconfig -add /vote01/ocr01
[root@ocfs1 vote01]# ocrconfig -add /vote02/ocr02
[root@ocfs1 vote01]# ocrconfig -add /vote03/ocr03

[root@ocfs1 vote03]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          4
         Total space (kbytes)     :     901284
         Used space (kbytes)      :      84848
         Available space (kbytes) :     816436
         ID                       : 1972206939
         Device/File Name         :      +VOTE
                                    Device/File integrity check succeeded
         Device/File Name         : /vote01/ocr01
                                    Device/File integrity check succeeded
         Device/File Name         : /vote02/ocr02
                                    Device/File integrity check succeeded
         Device/File Name         : /vote03/ocr03
                                    Device/File integrity check succeeded

                                    Device/File not configured

         PROT-607: More than one OCR storage type was detected.

         Cluster registry integrity check succeeded

         Logical corruption check succeeded
 
OCR 화일도 정상 추가 되었다.

▣ 남아 있는 ASM 정보를 삭제하자. 일단 삭제하려고 하는데 기본 백업위치가 ASM 이라 지워지지 않는다.
  
[root@ocfs1 vote03]# ocrconfig -delete +VOTE
PROT-48: The Oracle Cluster Registry location +VOTE cannot be deleted because the OCR backup location is an ASM disk group.

[root@ocfs1 vote03]# ocrconfig -showbackup

ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/backup00.ocr.258.1242819061     2505374838
ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/day.ocr.259.1242819065     2505374838
ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/week.ocr.260.1242819065     2505374838
PROT-25: Manual backups for the Oracle Cluster Registry are not available

[root@ocfs1 vote03]# ocrconfig -showbackuploc
The Oracle Cluster Registry backup location is [+VOTE].

▣ 백업 Location 을 새로 구성한 곳으로 지정해 준다.

[root@ocfs1 vote03]# ocrconfig -backuploc /vote01/
[root@ocfs1 vote03]# ocrconfig -showbackuploc
The Oracle Cluster Registry backup location is [/vote01/].

▣  이제 삭제가 된다.

[root@ocfs1 vote03]# ocrconfig -delete +VOTE
[root@ocfs1 vote03]# ocrconfig -showbackup

ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/backup00.ocr.258.1242819061     2505374838
ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/day.ocr.259.1242819065     2505374838
ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/week.ocr.260.1242819065     2505374838

▣ 수동으로 백업을 한번 받아준다.

[root@ocfs1 vote03]# ocrconfig -manualbackup
ocfs1     2026/09/01 12:40:40     /vote01/backup_20260901_124040.ocr     2505374838     

[root@ocfs1 vote03]# ocrconfig -showbackup

ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/backup00.ocr.258.1242819061     2505374838
ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/day.ocr.259.1242819065     2505374838
ocfs1     2026/09/01 11:31:03     +VOTE:/ocfs-cluster/OCRBACKUP/week.ocr.260.1242819065     2505374838
ocfs1     2026/09/01 12:40:40     /vote01/backup_20260901_124040.ocr     2505374838  

▣ 만일 이전 정보가 남아 있어서 깔끔하게 정리하고 싶다면 ASM 삭제전에 백업화일 들도 정리해 주자 (이건 안해봐서 모르겠다)

ocrconfig -delbackup +VOTE:/ocfs-cluster/OCRBACKUP/backup00.ocr.258.1242819061
ocrconfig -delbackup +VOTE:/ocfs-cluster/OCRBACKUP/day.ocr.259.1242819065
ocrconfig -delbackup +VOTE:/ocfs-cluster/OCRBACKUP/week.ocr.260.1242819065

▣ OCR 화일 확인

[root@ocfs1 vote03]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          4
         Total space (kbytes)     :     901284
         Used space (kbytes)      :      84880
         Available space (kbytes) :     816404
         ID                       : 1972206939
         Device/File Name         : /vote01/ocr01
                                    Device/File integrity check succeeded
         Device/File Name         : /vote02/ocr02
                                    Device/File integrity check succeeded
         Device/File Name         : /vote03/ocr03
                                    Device/File integrity check succeeded
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check succeeded
         Logical corruption check succeeded


▣ 이제 남아 있던  ASM Resource 의 흔적을 깨끗이 삭제해 준다.

srvctl update listener -listener ASMNET1LSNR_ASM -asm -remove -force
PRCR-1065 : Failed to stop resource ora.ASMNET1LSNR_ASM.lsnr
CRS-2670: Unable to start/relocate 'ora.asm_listener.type' because 'ora.asm' has a stop-time 'hard' dependency on it
CRS-5027: The ASM instance on node "ocfs1" cannot be stopped because it is currently being used by Cluster Ready Services.

-> 아직 ASM 에 정보가 남아서 인지 삭제되지 않는다. 재기동 해보자.  

crsctl stop crs -f
crsctl start crs

다시 진행

srvctl update listener -listener ASMNET1LSNR_ASM -asm -remove -force
srvctl stop asmnetwork
srvctl remove asmnetwork -all

srvctl stop diskgroup -g VOTE
srvctl remove diskgroup -diskgroup VOTE -force
srvctl stop asm -node ocfs1
srvctl stop asm -node ocfs2
srvctl remove asm -f

crsctl stop res ora.asm -init     
crsctl delete res ora.asm -init  <--- CRS daemon 도 기동되지 않게 삭제한다.

 

▣ 상태 확인
[root@ocfs1 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       ocfs1                    STABLE
               ONLINE  ONLINE       ocfs2                    STABLE
ora.chad
               ONLINE  ONLINE       ocfs1                    STABLE
               ONLINE  ONLINE       ocfs2                    STABLE
ora.helper
               OFFLINE OFFLINE      ocfs1                    IDLE,STABLE
               OFFLINE OFFLINE      ocfs2                    IDLE,STABLE
ora.net1.network
               ONLINE  ONLINE       ocfs1                    STABLE
               ONLINE  ONLINE       ocfs2                    STABLE
ora.ons
               ONLINE  ONLINE       ocfs1                    STABLE
               ONLINE  ONLINE       ocfs2                    STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.cdp1.cdp
      1        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.cvuhelper
      1        OFFLINE OFFLINE                               STABLE
ora.ocfs1.vip
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.ocfs2.vip
      1        ONLINE  ONLINE       ocfs2                    STABLE
ora.rhpserver
      1        OFFLINE OFFLINE                               STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       ocfs1                    STABLE
--------------------------------------------------------------------------------

[root@ocfs1 ~]# crsctl stat res -t -init
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cluster_interconnect.haip
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.crf
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.crsd
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.cssd
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.cssdmonitor
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.drivers.acfs
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.evmd
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.gipcd
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.gpnpd
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.mdnsd
      1        ONLINE  ONLINE       ocfs1                    STABLE
ora.storage
      1        ONLINE  ONLINE       ocfs1                    STABLE
--------------------------------------------------------------------------------

[root@ocfs1 ~]# df -h
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   4.0M     0  4.0M   0% /dev
tmpfs                      4.9G     0  4.9G   0% /dev/shm
tmpfs                      2.0G   13M  2.0G   1% /run
/dev/mapper/ol_ocfs1-root   11G  6.1G  4.9G  56% /
/dev/sda1                  960M  427M  534M  45% /boot
/dev/mapper/ol_ocfs1-u01    40G  5.0G   35G  13% /u01
/dev/sde                   2.0G  495M  1.6G  25% /vote03
/dev/sdd                   2.0G  495M  1.6G  25% /vote01
/dev/sdf                   2.0G  495M  1.6G  25% /vote02
/dev/sdc                    10G  551M  9.5G   6% /archive
/dev/sdb                    20G  1.1G   19G   6% /oradata
tmpfs                      995M   36K  994M   1% /run/user/0
tmpfs                      995M   36K  994M   1% /run/user/1001
oracle_clusterware         128M  4.7M  124M   4% /u01/app/oracle/crsdata/ocfs1/shm

GI SW 설치는 완료되었고 DB SW(과정 생략) 도 완료되었다.

 

다음은 dbca 로 Storage option 단계에서  /oradata 영역에 DB 를 생성해 보자.

 

기타 과정은 동일하며 생성 위치를 file system 으로 지정했을 시 정상적으로 구성이 됨을 확인 할수 있다.

 

단 이때 다른 CFS 환경과 마찬지로 spfile 은 /oradata 공유영역에 생성됨을 확인할수 있다.

 

SQL> show parameter pfile;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      /oradata/CDB/spfileCDB.ora

 

작업 끝~~

Posted by pat98

09-08 20:57
Flag Counter
Yesterday
Today
Total

글 보관함

최근에 올라온 글

달력

 « |  » 2026.9
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30

최근에 달린 댓글