OLR (Oracle Local Registry) stores the information used by OHASD during the startup sequence of the node.The OLR clusterware configuration file is present on all the nodes of the RAC database.It also stores the Oracle Restart’s metadata for Single instance database in 11gR2.
[[email protected] bin]$ ./ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 152 Available space (kbytes) : 261968 ID : 1434054292 Device/File Name : /u01/app/oracle/grid/11.2.0.2/db_1/cdata/localhost/anand-lab.olr Device/File integrity check succeeded Cluster registry integrity check succeeded Logical corruption check bypassed due to non-privileged user
To check for the logical corruption, login as root and do ocrcheck
[[email protected] bin]# ./ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 152 Available space (kbytes) : 261968 ID : 1434054292 Device/File Name : /u01/app/oracle/grid/11.2.0.2/db_1/cdata/localhost/anand-lab.olr Device/File integrity check succeeded Cluster registry integrity check succeeded Logical corruption check succeeded
If the OLR is corrupt or inaccessible, the OHAS doesn’t start.
[[email protected] bin]$ ./crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.dg ONLINE ONLINE anand-lab ora.DATA02.dg ONLINE ONLINE anand-lab ora.LISTENER.lsnr ONLINE ONLINE anand-lab ora.LISTENR_TEST.lsnr ONLINE ONLINE anand-lab ora.asm ONLINE ONLINE anand-lab Started ora.ons OFFLINE OFFLINE anand-lab -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.cssd 1 ONLINE ONLINE anand-lab ora.diskmon 1 ONLINE ONLINE anand-lab ora.evmd 1 ONLINE ONLINE anand-lab ora.matrix.db 1 ONLINE ONLINE anand-lab Open [[email protected] bin]$
[[email protected] bin]# ./ocrconfig -local -showbackup anand-lab 2011/04/29 14:42:14 /u01/app/oracle/grid/11.2.0.2/db_1/cdata/anand-lab/backup_20110429_144214.olr [[email protected] bin]# [[email protected] bin]# ./ocrconfig -local -manualbackup --> Took a new backup anand-lab 2011/08/20 12:18:06 /u01/app/oracle/grid/11.2.0.2/db_1/cdata/anand-lab/backup_20110820_121806.olr anand-lab 2011/04/29 14:42:14 /u01/app/oracle/grid/11.2.0.2/db_1/cdata/anand-lab/backup_20110429_144214.olr [[email protected] bin]#
Lets rename the OLR
[[email protected] localhost]$ ls -lrth -rw-r----- 1 oracle oinstall 261M Aug 20 12:18 anand-lab.olr [[email protected] localhost]$ mv anand-lab.olr anand-lab.olr.bkp [[email protected] localhost]$ ls -lrt -rw-r----- 1 oracle oinstall 272756736 Aug 20 12:18 anand-lab.olr.bkp
[[email protected] bin]$ ./ocrcheck -local PROTL-602: Failed to retrieve data from the local registry PROCL-26: Error while accessing the physical storage Operating System error [No such file or directory] [2]
[[email protected] client]$ pwd /u01/app/oracle/grid/11.2.0.2/db_1/log/anand-lab/client [[email protected] client]$ cat ocrcheck_4189.log Oracle Database 11g Clusterware Release 11.2.0.2.0 - Production Copyright 1996, 2010 Oracle. All rights reserved. 2011-08-20 12:29:39.216: [OCRCHECK][3086636736]ocrcheck starts... 2011-08-20 12:29:39.217: [ OCROSD][3086636736]utopen:6m': failed in stat OCR file/disk /u01/app/oracle/grid/11.2.0.2/db_1/cdata/localhost/anand-lab.olr, errno=2, os err string=No such file or directory 2011-08-20 12:29:39.217: [ OCROSD][3086636736]utopen:7: failed to open any OCR file/disk, errno=2, os err string=No such file or directory 2011-08-20 12:29:39.217: [ OCRRAW][3086636736]proprinit: Could not open raw device 2011-08-20 12:29:39.217: [ default][3086636736]a_init:7!: Backend init unsuccessful : [26] 2011-08-20 12:29:39.217: [OCRCHECK][3086636736]Failed to access OCR repository. Retval [26] Error [PROCL-26: Error while accessing the physical storage Operating System error [No such file or directory] [2]] 2011-08-20 12:29:39.217: [OCRCHECK][3086636736]Failed to initialize ocrchek2 2011-08-20 12:29:39.217: [OCRCHECK][3086636736]Exiting [status=failed]...
Stopped the OHASD using crsctl stop has and started back –
[[email protected] bin]# ./crsctl start has CRS-4563: Insufficient user privileges. CRS-4000: Command Start failed, or completed with errors.
[[email protected] localhost]# touch anand-lab.olr [[email protected] localhost]# [[email protected] localhost]# ls -lrt -rw-r----- 1 oracle oinstall 272756736 Aug 20 12:34 anand-lab.olr.bkp -rw-r--r-- 1 root root 0 Aug 20 12:45 anand-lab.olr [[email protected] localhost]# chown oracle:oinstall anand-lab.olr [[email protected] localhost]# chmod 640 anand-lab.olr [[email protected] localhost]# ls -lrt total 2828 -rw-r----- 1 oracle oinstall 272756736 Aug 20 12:34 anand-lab.olr.bkp -rw-r----- 1 oracle oinstall 0 Aug 20 12:45 anand-lab.olr [[email protected] localhost]#
Restore OLR from backup –
[[email protected] bin]# ./ocrconfig -local -restore /u01/app/oracle/grid/11.2.0.2/db_1/cdata/anand-lab/backup_20110820_121806.olr [[email protected] bin]#
Log shows –
[[email protected] client]$ more ocrconfig_5059.log Oracle Database 11g Clusterware Release 11.2.0.2.0 - Production Copyright 1996, 2010 Oracle. All rights reserved. 2011-08-20 13:05:07.580: [ OCRCONF][3086165696]ocrconfig starts... 2011-08-20 13:05:07.580: [ OCRCONF][3086165696]Started to restore the Oracle Local Registry location from backup file [/u01/app/oracle/grid/11.2.0.2/db_1/cdata/anand-lab/backup_20110820_121806.olr]. 2011-08-20 13:05:07.631: [ OCRMSG][3086165696]prom_waitconnect: CONN NOT ESTABLISHED (0,29,1,2) 2011-08-20 13:05:07.631: [ OCRMSG][3086165696]GIPC error [29] msg [gipcretConnectionRefused] 2011-08-20 13:05:07.632: [ OCRMSG][3086165696]prom_connect: error while waiting for connection complete [24] 2011-08-20 13:05:07.927: [ OCRCONF][3086165696]Successfully restored OCR and set block 0 2011-08-20 13:05:07.927: [ OCRCONF][3086165696]Exiting [status=success]...
[[email protected] bin]# ./crsctl stat res -t CRS-4639: Could not contact Oracle High Availability Services CRS-4000: Command Status failed, or completed with errors. Time to start OHASd [[email protected] bin]# ./crsctl start has CRS-4123: Oracle High Availability Services has been started. [[email protected] bin]#
[[email protected] bin]# ./crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.dg ONLINE ONLINE anand-lab ora.DATA02.dg ONLINE ONLINE anand-lab ora.LISTENER.lsnr ONLINE ONLINE anand-lab ora.LISTENR_TEST.lsnr ONLINE ONLINE anand-lab ora.asm ONLINE ONLINE anand-lab Started ora.ons OFFLINE OFFLINE anand-lab -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.cssd 1 ONLINE ONLINE anand-lab ora.diskmon 1 ONLINE ONLINE anand-lab ora.evmd 1 ONLINE ONLINE anand-lab ora.matrix.db 1 ONLINE ONLINE anand-lab Open [[email protected] bin]#
2 thoughts on “Restoring OLR – 11gR2”
Anand, thank you so much for this article. It helped me to troubleshoot an issue that I was having while upgrading clusterware software from version 12.1.0.1.0 to 12.1.0.2.0.
Thanks Jerry. Good to know it was useful 🙂