[RAC][INSTALL] 3_Grid 엔진 설치

설치파일 업로드 및 압축 해제 (RAC1)

반드시 rac2의 ORACLE_HOME(=GRID_HOME)에는 모든 내용이 비어있어야지 설치 가능하다. 만약 비어있지 않고 파일들이 존재할 경우 GRID 설치 중 에러가 발생한다.

[root@ol79rac1 tmp]# ls -lrt | grep Oracle\ Database\
-rw-r--r--. 1 root root 3453696911 Dec 27 16:50 Oracle Database 12.2.0.1.0 for Linux x86-64, 3.2 GB.zip
> oracle engine 설치를 위한 파일
-rw-r--r--. 1 root root 2994687209 Dec 27 16:54 Oracle Database Grid Infrastructure 12.2.0.1.0 for Linux x86-64, 2.8 GB.zip
> grid 설치를 위한 파일

grid@ol79rac1[+ASM1]:/tmp$ cp Oracle\ Database\ Grid\ Infrastructure\ 12.2.0.1.0\ for\ Linux\ x86-64\,\ 2.8\ GB.zip -d $GRID_HOME

grid@ol79rac1[+ASM1]:/tmp$ cd $GRID_HOME
grid@ol79rac1[+ASM1]:/grid_engine/12c/grid$ ls -la
total 2924500
drwxr-xr-x. 2 grid dba         89 Dec 27 16:56 .
drwxr-xr-x. 3 grid dba         18 Dec 27 16:13 ..
-rw-r--r--. 1 grid dba 2994687209 Dec 27 16:56 Oracle Database Grid Infrastructure 12.2.0.1.0 for Linux x86-64, 2.8 GB.zip

grid@ol79rac1[+ASM1]:/grid_engine/12c/grid$ unzip Oracle\ Database\ Grid\ Infrastructure\ 12.2.0.1.0\ for\ Linux\ x86-64\,\ 2.8\ GB.zip

Archive:  Oracle Database Grid Infrastructure 12.2.0.1.0 for Linux x86-64, 2.8 GB.zip
   creating: addnode/
  inflating: addnode/addnode_oraparam.ini.sbs
  inflating: addnode/addnode.pl
  inflating: addnode/addnode.sh
  inflating: addnode/addnode_oraparam.ini
...

 

SSH 접속 설정 (RAC1)

SSHKey를 설정하는 이유는 서버에서 접속할 때 비밀번호 대신 key를 제출하는 방법으로 RAC 설치 시, 1번에서 설치되는 내역을 SSH를 통해 2번으로 파일을 넘겨주기때문에 필수적으로 설정 필요하다

경로 : $GRID_HOME/oui/prov/resources/scripts
key설치 : ./sshUserSetup.sh -user grid -hosts "<rac1,2 hostname>" -noPromptPassphrase -advanced

grid@ol79rac1[+ASM1]:/home/grid$ cd $GRID_HOME/oui/prov/resources/scripts

grid@ol79rac1[+ASM1]:/grid_engine/12c/grid/oui/prov/resources/scripts$ ls -al
total 112
drwxr-x---. 2 grid dba  4096 Jan 27  2017 .
drwxr-xr-x. 3 grid dba  4096 Jan 27  2017 ..
-rwxr-x---. 1 grid dba  6983 Jul 20  2006 ouiSetup
-rwxr-x---. 1 grid dba 16520 Apr  1  2009 sshConnectivity.sh
-rwxr-x---. 1 grid dba 35064 Dec 12  2006 sshUserSetupNT.sh
-rwxr-x---. 1 grid dba 33333 Jan  7  2015 sshUserSetup.sh --ssh user 설치 쉘
-rwxr-x---. 1 grid dba  1394 Jul 20  2006 validatePaths

grid@ol79rac1[+ASM1]:/grid_engine/12c/grid/oui/prov/resources/scripts$ ./sshUserSetup.sh -user grid -hosts "ol79rac1 ol79rac2" -noPromptPassphrase -advanced

 

 

The output of this script is also logged into /tmp/sshUserSetup_2022-12-27-18-53-26.log
Hosts are ol79rac1 ol79rac2
user is grid
Platform:- Linux
Checking if the remote hosts are reachable
PING ol79rac1 (192.168.56.101) 56(84) bytes of data.
..

NOTE 1:
..

NOTE 2:
..
>>입력 필요
Do you want to continue and let the script make the above mentioned changes (yes/no)?
yes

User chose to skip passphrase related questions.
Creating .ssh directory on local host, if not present already
..

The key's randomart image is:
+---[RSA 1024]----+
|  ++.*o++..      |
| E.oBo*o.. .     |
|+o++ ++o. .      |
|=*o...oo..       |
|=+=..o..S .      |
|oB.. ..  o       |
|+ o              |
|..               |
|..               |
+----[SHA256]-----+
Creating .ssh directory and setting permissions on remote host ol79rac1
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR grid. THIS IS AN SSH REQUIREMENT.
The script would create ~grid/.ssh/config file on remote host ol79rac1. If a config file exists already at ~grid/.ssh/config, it would be backed up to ~grid/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host ol79rac1.
Warning: Permanently added 'ol79rac1,192.168.56.101' (ECDSA) to the list of known hosts.
>>입력 필요
grid@ol79rac1's password: grid

Done with creating .ssh directory and setting permissions on remote host ol79rac1.
Creating .ssh directory and setting permissions on remote host ol79rac2
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR grid. THIS IS AN SSH REQUIREMENT.
The script would create ~grid/.ssh/config file on remote host ol79rac2. If a config file exists already at ~grid/.ssh/config, it would be backed up to ~grid/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host ol79rac2.
Warning: Permanently added 'ol79rac2,192.168.56.102' (ECDSA) to the list of known hosts.
>>입력 필요
grid@ol79rac2's password: grid

Done with creating .ssh directory and setting permissions on remote host ol79rac2.
Copying local host public key to the remote host ol79rac1
The user may be prompted for a password or passphrase here since the script would be using SCP for host ol79rac1.
>>입력 필요
grid@ol79rac1's password: grid
Done copying local host public key to the remote host ol79rac1
Copying local host public key to the remote host ol79rac2
The user may be prompted for a password or passphrase here since the script would be using SCP for host ol79rac2.
>>입력 필요
grid@ol79rac2's password: grid

Done copying local host public key to the remote host ol79rac2
Creating keys on remote host ol79rac1 if they do not exist already. This is required to setup SSH on host ol79rac1.

Creating keys on remote host ol79rac2 if they do not exist already. This is required to setup SSH on host ol79rac2.
Generating public/private rsa key pair.
Your identification has been saved in .ssh/id_rsa.
Your public key has been saved in .ssh/id_rsa.pub.
The key fingerprint is:
SHA256:ku9Wzz7Lar+jDE8KWZIMfLjINlmccr39W+aBljH+opA grid@ol79rac2
The key's randomart image is:
+---[RSA 1024]----+
|                 |
|   o +           |
|  . O o          |
| . * = =         |
|  * . B S o      |
| . .   B o.=     |
|      E o.Bo+    |
|       +.BoB=.   |
|       .+o**O=   |
+----[SHA256]-----+
Updating authorized_keys file on remote host ol79rac1
Updating known_hosts file on remote host ol79rac1
Updating authorized_keys file on remote host ol79rac2
Updating known_hosts file on remote host ol79rac2
cat: /home/grid/.ssh/known_hosts.tmp: No such file or directory
cat: /home/grid/.ssh/authorized_keys.tmp: No such file or directory
SSH setup is complete.

------------------------------------------------------------------------
Verifying SSH setup
===================
The script will now run the date command on the remote nodes using ssh
to verify if ssh is setup correctly. IF THE SETUP IS CORRECTLY SETUP,
THERE SHOULD BE NO OUTPUT OTHER THAN THE DATE AND SSH SHOULD NOT ASK FOR
PASSWORDS. If you see any output other than date or are prompted for the
password, ssh is not setup correctly and you will need to resolve the
issue and set up ssh again.
The possible causes for failure could be:
1. The server settings in /etc/ssh/sshd_config file do not allow ssh
for user grid.
2. The server may have disabled public key based authentication.
3. The client public key on the server may be outdated.
4. ~grid or ~grid/.ssh on the remote host may not be owned by grid.
5. User may not have passed -shared option for shared remote users or
may be passing the -shared option for non-shared remote users.
6. If there is output in addition to the date, but no password is asked,
it may be a security alert shown as part of company policy. Append the
additional text to the <OMS HOME>/sysman/prov/resources/ignoreMessages.txt file.
------------------------------------------------------------------------
--ol79rac1:--
Running /usr/bin/ssh -x -l grid ol79rac1 date to verify SSH connectivity has been setup from local host to ol79rac1.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Tue Dec 27 18:56:59 KST 2022
------------------------------------------------------------------------
--ol79rac2:--
Running /usr/bin/ssh -x -l grid ol79rac2 date to verify SSH connectivity has been setup from local host to ol79rac2.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Tue Dec 27 18:56:56 KST 2022
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from ol79rac1 to ol79rac1
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Tue Dec 27 18:57:00 KST 2022
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from ol79rac1 to ol79rac2
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Tue Dec 27 18:56:57 KST 2022
------------------------------------------------------------------------
-Verification from complete-
SSH verification complete.

ssh 정상적으로 맺었는지 확인
 - remote node 접속 시 패스워드 관련하여 물어보지 않아야함

#외부 통신 영역
grid@ol79rac1[+ASM1]:/home/grid$ ssh ol79rac2 date
Tue Dec 27 19:01:27 KST 2022

#내부 interconnect 영역
grid@ol79rac1[+ASM1]:/home/grid$ ssh ol79rac2-priv date

>>ssh 설정되어있지 않다면 아래와 같이 수동 설정 필요
The authenticity of host 'ol79rac2-priv (10.0.2.11)' can't be established.
ECDSA key fingerprint is SHA256:qxWEbXk3xSYWEgLYJ/5b4r6Lx3OG/pD5rUKjtxq5Z2U.
ECDSA key fingerprint is MD5:23:c7:f9:ed:ff:7f:da:cb:5e:ee:9a:3c:e8:55:66:fa.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ol79rac2-priv,10.0.2.11' (ECDSA) to the list of known hosts.
Tue Dec 27 19:01:45 KST 2022

 

cvuqdisk 패키지 설치 (RAC1,RAC2)

모든 모드에 해당 패키지가 없으면 클러스터 확인 유틸리티(CVU)는 공유 디스크를 찾을 수가 없다 

반드시 root 계정에서 수행해야하며 oinstall이라는 그룹이 필요하다.

경로 : $GRID_HOME/cv/rpm

[root@ol79rac1 tmp]# cd /grid_engine/12c/grid/cv/rpm

[root@ol79rac1 rpm]# ls -l
total 12
-rw-r--r--. 1 grid dba 8860 Jan  5  2017 cvuqdisk-1.0.10-1.rpm

[root@ol79rac1 rpm]# rpm -ivh cvuqdisk-1.0.10-1.rpm
Preparing...                          ################################# [100%]
Using default group oinstall to install package
Updating / installing...
   1:cvuqdisk-1.0.10-1                ################################# [100%]

------------------------------------------------------------------------------
1. oinstall이라는 group이 없을 경우 발생

>> 아래와 같이 failed 라면 oinstall이라는 그룹 필요
[root@ol79rac1 rpm]# rpm -ivh cvuqdisk-1.0.10-1.rpm
Preparing...                          ################################# [100%]
Using default group oinstall to install package
Group oinstall not found
oinstall : Group doesn't exist.
Please define environment variable CVUQDISK_GRP with the correct group to be used
error: %pre(cvuqdisk-1.0.10-1.x86_64) scriptlet failed, exit status 1
error: cvuqdisk-1.0.10-1.x86_64: install failed

-----------------------------------------------------------------------------------
2. root가 아닌 grid에서 해당 패키지를 설치 할 경우 발생
grid@ol79rac1[+ASM1]:/grid_engine/12c/grid/cv/rpm$ rpm -ivh cvuqdisk-1.0.10-1.rpm
error: can't create transaction lock on /var/lib/rpm/.rpm.lock (Permission denied)

 

설치 전 사전 점검 (RAC1,RAC2)

 

runcluvfy.log

0.03MB

해당 부분 확인을 위해선 /tmp 디렉토리가 5G이상이어야하며 그렇지 않을 경우 /에 여유공간 필요 (필수사항x)

경로 : $GRID_HOME
파일명 : runcluvfy.sh

grid@ol79rac1[+ASM1]:/home/grid$ cd $GRID_HOME
grid@ol79rac1[+ASM1]:/grid_engine/12c/grid$ 

grid@ol79rac1[+ASM1]:/grid_engine/12c/grid$ ls -l run*
-rwxr-x---. 1 grid dba 628 Sep  5  2015 runcluvfy.sh

grid@ol79rac1[+ASM1]:/grid_engine/12c/grid$ ./runcluvfy.sh stage -pre crsinst -n ol79rac1,ol79rac2 -osdba dba -orainv dba -verbose
>>shell 수행 내역은 상단에 첨부된 log 확인

 

Grid 설치

경로 : $GRID_HOME
파일 : gridSetup.

grid@ol79rac1[+ASM1]:/grid_engine/12c/grid$ ./gridSetup.sh

위와 같이 normal로 선택할 경우 적어도 3개의 디스크가 필요로 하여 설정한 디스크 갯수는 2개 뿐이어서 "Normal -> External"로 변경 후 설치 진행해야한다. 

ORACLE_BASE 및 HOME 설정 관련 내역으로 HOME 및 BASE는 반드시 디렉토리가 분리되어 있어야하며 RAC2은 디렉토리 생성만 진행하고 내부에 디렉토리나 스크립트 파일들이 하나라도 존재할경우 다음단계로 넘어갈 때 에러가 발생한다

다른 에러들은 무시해도 되지만 shmmax만큼은 확인 후 수정하고 설치를 진행해야한다. 변경해야하는 값 관련해서는 more details를 선택하면 상세내용을 확인할 수 있다.

# vi /etc/sysctl.conf

kernel.shmmax = 5220685824 -> 5220683776 으로 변경 후 설치 진행

※ 순서 중요 (1번 진행 후 -> 2번 진행)

1. orainstRoot.sh

[root@ol79rac1 ~]# /grid_engine/oraInventory/orainstRoot.sh
▼
[root@ol79rac2 ~]# /grid_engine/oraInventory/orainstRoot.sh

2. root.sh

[root@ol79rac1 ~]# /grid_engine/12c/grid/root.sh
▼
[root@ol79rac2 ~]# /grid_engine/12c/grid/root.sh

[root@ol79rac1 ~]# /grid_engine/12c/grid/root.sh

Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /grid_engine/12c/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /grid_engine/12c/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /grid_engine/base/crsdata/ol79rac1/crsconfig/rootcrs_ol79rac1_2022-12-28_03-08-24PM.log
2022/12/28 15:08:28 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2022/12/28 15:08:28 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2022/12/28 15:09:10 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2022/12/28 15:09:10 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2022/12/28 15:09:14 CLSRSC-363: User ignored prerequisites during installation
2022/12/28 15:09:14 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2022/12/28 15:09:15 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2022/12/28 15:09:16 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2022/12/28 15:09:21 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2022/12/28 15:09:22 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2022/12/28 15:09:22 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2022/12/28 15:09:44 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2022/12/28 15:09:51 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2022/12/28 15:09:51 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2022/12/28 15:09:55 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2022/12/28 15:10:10 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2022/12/28 15:10:29 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2022/12/28 15:10:33 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol79rac1'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol79rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2022/12/28 15:10:53 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2022/12/28 15:10:57 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol79rac1'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol79rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'ol79rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'ol79rac1'
CRS-2676: Start of 'ora.mdnsd' on 'ol79rac1' succeeded
CRS-2676: Start of 'ora.evmd' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ol79rac1'
CRS-2676: Start of 'ora.gpnpd' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol79rac1'
CRS-2672: Attempting to start 'ora.gipcd' on 'ol79rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'ol79rac1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ol79rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'ol79rac1'
CRS-2676: Start of 'ora.diskmon' on 'ol79rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ol79rac1' succeeded

Disk groups created successfully. Check /grid_engine/base/cfgtoollogs/asmca/asmca-221228PM031134.log for details.


2022/12/28 15:12:13 CLSRSC-482: Running command: '/grid_engine/12c/grid/bin/ocrconfig -upgrade grid dba'
CRS-2672: Attempting to start 'ora.crf' on 'ol79rac1'
CRS-2672: Attempting to start 'ora.storage' on 'ol79rac1'
CRS-2676: Start of 'ora.storage' on 'ol79rac1' succeeded
CRS-2676: Start of 'ora.crf' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ol79rac1'
CRS-2676: Start of 'ora.crsd' on 'ol79rac1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 3fb1c164392b4faabf175f78cea18a67.
Successfully replaced voting disk group with +OCR_VOTE.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   3fb1c164392b4faabf175f78cea18a67 (/dev/asm/orc01) [OCR_VOTE]
Located 1 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol79rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'ol79rac1'
CRS-2677: Stop of 'ora.crsd' on 'ol79rac1' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'ol79rac1'
CRS-2673: Attempting to stop 'ora.crf' on 'ol79rac1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'ol79rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'ol79rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'ol79rac1' succeeded
CRS-2677: Stop of 'ora.crf' on 'ol79rac1' succeeded
CRS-2677: Stop of 'ora.storage' on 'ol79rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'ol79rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'ol79rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'ol79rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ol79rac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ol79rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'ol79rac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'ol79rac1'
CRS-2677: Stop of 'ora.evmd' on 'ol79rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'ol79rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'ol79rac1'
CRS-2677: Stop of 'ora.cssd' on 'ol79rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'ol79rac1'
CRS-2677: Stop of 'ora.gipcd' on 'ol79rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol79rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2022/12/28 15:13:32 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.evmd' on 'ol79rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'ol79rac1'
CRS-2676: Start of 'ora.mdnsd' on 'ol79rac1' succeeded
CRS-2676: Start of 'ora.evmd' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ol79rac1'
CRS-2676: Start of 'ora.gpnpd' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'ol79rac1'
CRS-2676: Start of 'ora.gipcd' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol79rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ol79rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'ol79rac1'
CRS-2676: Start of 'ora.diskmon' on 'ol79rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ol79rac1'
CRS-2672: Attempting to start 'ora.ctssd' on 'ol79rac1'
CRS-2676: Start of 'ora.ctssd' on 'ol79rac1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ol79rac1'
CRS-2676: Start of 'ora.asm' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'ol79rac1'
CRS-2676: Start of 'ora.storage' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'ol79rac1'
CRS-2676: Start of 'ora.crf' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ol79rac1'
CRS-2676: Start of 'ora.crsd' on 'ol79rac1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: ol79rac1
CRS-6016: Resource auto-start has completed for server ol79rac1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2022/12/28 15:15:32 CLSRSC-343: Successfully started Oracle Clusterware stack
2022/12/28 15:15:32 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'ol79rac1'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ol79rac1'
CRS-2676: Start of 'ora.asm' on 'ol79rac1' succeeded
CRS-2672: Attempting to start 'ora.OCR_VOTE.dg' on 'ol79rac1'
CRS-2676: Start of 'ora.OCR_VOTE.dg' on 'ol79rac1' succeeded
2022/12/28 15:18:37 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.



2022/12/28 15:20:02 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

[root@ol79rac2 ~]# /grid_engine/12c/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /grid_engine/12c/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /grid_engine/12c/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /grid_engine/base/crsdata/ol79rac2/crsconfig/rootcrs_ol79rac2_2022-12-28_04-25-14PM.log
2022/12/28 16:25:17 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2022/12/28 16:25:17 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2022/12/28 16:25:43 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2022/12/28 16:25:43 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2022/12/28 16:25:44 CLSRSC-363: User ignored prerequisites during installation
2022/12/28 16:25:44 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2022/12/28 16:25:45 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2022/12/28 16:25:45 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2022/12/28 16:25:46 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2022/12/28 16:25:46 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2022/12/28 16:25:47 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2022/12/28 16:25:48 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2022/12/28 16:25:49 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2022/12/28 16:25:49 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2022/12/28 16:25:50 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2022/12/28 16:26:05 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2022/12/28 16:26:23 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2022/12/28 16:26:24 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol79rac2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol79rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2022/12/28 16:26:51 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2022/12/28 16:26:52 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol79rac2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol79rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol79rac2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol79rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2022/12/28 16:27:11 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.evmd' on 'ol79rac2'
CRS-2672: Attempting to start 'ora.mdnsd' on 'ol79rac2'
CRS-2676: Start of 'ora.mdnsd' on 'ol79rac2' succeeded
CRS-2676: Start of 'ora.evmd' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ol79rac2'
CRS-2676: Start of 'ora.gpnpd' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'ol79rac2'
CRS-2676: Start of 'ora.gipcd' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol79rac2'
CRS-2676: Start of 'ora.cssdmonitor' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ol79rac2'
CRS-2672: Attempting to start 'ora.diskmon' on 'ol79rac2'
CRS-2676: Start of 'ora.diskmon' on 'ol79rac2' succeeded
CRS-2676: Start of 'ora.cssd' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ol79rac2'
CRS-2672: Attempting to start 'ora.ctssd' on 'ol79rac2'
CRS-2676: Start of 'ora.ctssd' on 'ol79rac2' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ol79rac2'
CRS-2676: Start of 'ora.asm' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'ol79rac2'
CRS-2676: Start of 'ora.storage' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'ol79rac2'
CRS-2676: Start of 'ora.crf' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ol79rac2'
CRS-2676: Start of 'ora.crsd' on 'ol79rac2' succeeded
CRS-6017: Processing resource auto-start for servers: ol79rac2
CRS-2672: Attempting to start 'ora.net1.network' on 'ol79rac2'
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'ol79rac2'
CRS-2676: Start of 'ora.net1.network' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.ons' on 'ol79rac2'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'ol79rac2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ol79rac2'
CRS-2676: Start of 'ora.ons' on 'ol79rac2' succeeded
CRS-2676: Start of 'ora.asm' on 'ol79rac2' succeeded
CRS-6016: Resource auto-start has completed for server ol79rac2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2022/12/28 16:29:16 CLSRSC-343: Successfully started Oracle Clusterware stack
2022/12/28 16:29:16 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2022/12/28 16:29:30 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.

2022/12/28 16:29:40 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

 

Grid 상태 확인

- ora.ons ol79ra1 에만 offline인 것은 현재 해결중

grid@ol79rac1[+ASM1]:/home/grid$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       ol79rac1                 STABLE
               ONLINE  ONLINE       ol79rac2                 STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       ol79rac1                 STABLE
               ONLINE  ONLINE       ol79rac2                 STABLE
ora.MGMT.dg
               ONLINE  ONLINE       ol79rac1                 STABLE
               ONLINE  ONLINE       ol79rac2                 STABLE
ora.OCR_VOTE.dg
               ONLINE  ONLINE       ol79rac1                 STABLE
               ONLINE  ONLINE       ol79rac2                 STABLE
ora.chad
               ONLINE  ONLINE       ol79rac1                 STABLE
               ONLINE  ONLINE       ol79rac2                 STABLE
ora.net1.network
               ONLINE  ONLINE       ol79rac1                 STABLE
               ONLINE  ONLINE       ol79rac2                 STABLE
ora.ons
               ONLINE  ONLINE       ol79rac1                 STABLE
               ONLINE  ONLINE       ol79rac2                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       ol79rac1                 STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       ol79rac1                 169.254.122.197 10.0
                                                             .2.10,STABLE
ora.asm
      1        ONLINE  ONLINE       ol79rac1                 Started,STABLE
      2        ONLINE  ONLINE       ol79rac2                 Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       ol79rac1                 STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       ol79rac1                 Open,STABLE
ora.ol79rac1.vip
      1        ONLINE  ONLINE       ol79rac1                 STABLE
ora.ol79rac2.vip
      1        ONLINE  ONLINE       ol79rac2                 STABLE
ora.qosmserver
      1        ONLINE  ONLINE       ol79rac1                 STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       ol79rac1                 STABLE
--------------------------------------------------------------------------------

 

Grid Process 확인

grid@ol79rac1[+ASM1]:/home/grid$ ps -ef | grep d.bin

root       356     1  0 04:58 ?        00:00:20 /grid_engine/12c/grid/bin/osysmond.bin
root       370     1  0 04:58 ?        00:00:14 /grid_engine/12c/grid/bin/crsd.bin reboot
root       536     1  0 04:58 ?        00:00:16 /grid_engine/12c/grid/bin/ologgerd -M
grid       629     1  0 04:58 ?        00:00:17 /grid_engine/12c/grid/bin/oraagent.bin
root       642     1  0 04:58 ?        00:00:12 /grid_engine/12c/grid/bin/orarootagent.bin
grid       774     1  0 04:58 ?        00:00:00 /grid_engine/12c/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit
grid       786     1  0 04:58 ?        00:00:02 /grid_engine/12c/grid/bin/scriptagent.bin
grid      1001     1  0 04:58 ?        00:00:00 /grid_engine/12c/grid/bin/tnslsnr LISTENER -no_crs_notify -inherit
grid      1005     1  0 04:58 ?        00:00:00 /grid_engine/12c/grid/bin/tnslsnr LISTENER_SCAN1 -no_crs_notify -inherit
grid      1009     1  0 04:58 ?        00:00:00 /grid_engine/12c/grid/bin/tnslsnr MGMTLSNR -no_crs_notify -inherit
root      3347   356  0 04:59 ?        00:00:03 /grid_engine/12c/grid/perl/bin/perl /grid_engine/12c/grid/bin/diagsnap.pl start
root     32112     1  0 04:57 ?        00:00:08 /grid_engine/12c/grid/bin/ohasd.bin reboot
grid     32114 32033  0 05:49 pts/0    00:00:00 grep --color=auto d.bin
root     32186     1  0 04:57 ?        00:00:04 /grid_engine/12c/grid/bin/orarootagent.bin
grid     32260     1  0 04:57 ?        00:00:06 /grid_engine/12c/grid/bin/oraagent.bin
grid     32280     1  0 04:57 ?        00:00:02 /grid_engine/12c/grid/bin/mdnsd.bin
grid     32281     1  0 04:57 ?        00:00:08 /grid_engine/12c/grid/bin/evmd.bin
grid     32331     1  0 04:57 ?        00:00:02 /grid_engine/12c/grid/bin/gpnpd.bin
grid     32359 32281  0 04:57 ?        00:00:02 /grid_engine/12c/grid/bin/evmlogger.bin -o /grid_engine/12c/grid/log/[HOSTNAME]/evmd/evmlogger.info -l /grid_engine/12c/grid/log/[HOSTNAME]/evmd/evmlogger.log
grid     32372     1  0 04:57 ?        00:00:06 /grid_engine/12c/grid/bin/gipcd.bin
root     32410     1  0 04:57 ?        00:00:03 /grid_engine/12c/grid/bin/cssdmonitor
root     32428     1  0 04:57 ?        00:00:03 /grid_engine/12c/grid/bin/cssdagent
grid     32441     1  0 04:57 ?        00:00:12 /grid_engine/12c/grid/bin/ocssd.bin
root     32615     1  0 04:57 ?        00:00:08 /grid_engine/12c/grid/bin/octssd.bin reboot

 

DiskGroup 설치

grid@ol79rac1[+ASM1]:/home/grid$ asmca

 

 

※ DATA 영역 생성

 

※ RECOVERY 영역 생성

<Diskgroup 생성 조회>

grid@ol79rac1[+ASM1]:/home/grid$ asmcmd
ASMCMD> ls -l
State    Type    Rebal  Name
MOUNTED  EXTERN  N      DATA/
MOUNTED  EXTERN  N      MGMT/
MOUNTED  EXTERN  N      OCR_VOTE/
MOUNTED  EXTERN  N      RECO/