Oracle11gRAC数据库节点损坏恢复.docx

1、Oracle11gRAC数据库节点损坏恢复 Oracle11gR2 Rac节点损坏恢复方正国际公共安全事业部技术文档1.文档属性文档属性内容文档名称Oracle11gR2 Rac 节点损坏恢复报告文档版本号A1文档状态正式稿文档编写完成日期2016年9月14日作者2.文档变更历史清单文件版本号修正日期修正人备注3.本次修改变更说明序号变更内容简述1.2.3.4.5.第1章 概述#省厅警务综合平台项目中,锦州数据库现场的用户因为在未停止数据库的情况下,扩展磁盘阵列,更换磁盘,导致1节点本地磁盘阵列挂不上,导致只能一个节点使用数据库,需要重新配置oracle集群信息.第2章 系统环境项目名称 服务

2、器名RAC节点1RAC节点2操作系统RedHat 6RedHat 6集群软件Oracle GRIDOracle GRID服务器主机名his1his2IP 地址示例172.172. 系统用户用户名RootGridoracle系统组dbaasmdbaasmadminoinstallasmoperdbaasmdbaasmadminoinstallasmoper第3章 数据库环境项目名称 服务器名称RAC节点1RAC 节点2公共IP地址172.172.虚拟IP地址172.172.心跳IP地址1ScanIP 地址172.Oracle RAC SIDorcl1orcl2数据库名称orcl数据文件路径+DA

3、TA归档文件+ARCH数据库版本Oracle Database 11g Enterprise Edition Release 11.2.0.4.0GRID_BASE目录 /u01/app/gridGRID_HOME目录ORACLE_BASE目录/u01/app/oracleORACLE_HOME目录/u01/app/oracle 数据库监听端口1521数据库字符集ZHS16GBK数据库用户sys,system密码oracle数据库硬盘管理方式ASM第4章 实施步骤4.1 操作系统准备工作包括如下操作,主要和rac安装文档相同,如果需要重新安装操作系统,请参考rac安装文档的操作,进行操作系统配

4、置.下面只列出：4.1.1 配置grid用户与oracle用户等效性Grid用户在his1上：su - grid$ mkdir /.ssh$ chmod 700 /.ssh$ ssh-keygen -t rsa$ ssh-keygen -t dsa$ cat /.ssh/id_rsa.pub /.ssh/authorized_keys$ cat /.ssh/id_dsa.pub /.ssh/authorized_keys在his2上：$ mkdir /.ssh$ chmod 700 /.ssh$ ssh-keygen -t rsa$ ssh-keygen -t dsa在his1上：$ ssh

5、jkkdb2 cat /.ssh/id_rsa.pub /.ssh/authorized_keys$ ssh jkkdb2 cat /.ssh/id_dsa.pub /.ssh/authorized_keys$ scp /.ssh/authorized_keys jkkdb2:/.ssh/authorized_keys验证$ ssh his1 date$ ssh his1-priv date$ ssh his2 date$ ssh his2-priv dateOracle用户在his1上：su - grid$ mkdir /.ssh$ chmod 700 /.ssh$ ssh-keygen -

6、t rsa$ ssh-keygen -t dsa$ cat /.ssh/id_rsa.pub /.ssh/authorized_keys$ cat /.ssh/id_dsa.pub /.ssh/authorized_keys在his2上：$ mkdir /.ssh$ chmod 700 /.ssh$ ssh-keygen -t rsa$ ssh-keygen -t dsa在his1上：$ ssh jkkdb2 cat /.ssh/id_rsa.pub /.ssh/authorized_keys$ ssh jkkdb2 cat /.ssh/id_dsa.pub /.ssh/authorized_

7、keys$ scp /.ssh/authorized_keys jkkdb2:/.ssh/authorized_keys验证$ ssh his1 date$ ssh his1-priv date$ ssh his2 date$ ssh his2-priv date4.1.2 设置raw,磁盘阵列联系硬件厂家,扩展磁盘操作等等,两个机器的磁盘信息相同,主要是挂的磁盘号要一致.4.1.3 重启his1服务器4.2 停止原来1节点vip$ srvctl disable listener -l listener_name -n his1$ srvctl stop listener -l listene

8、r_name -n his14.3 删除原1节点数据库实例信息oracle用户执行：$dbca -silent -deleteInstance -nodeList his1 -gdbName orcl -instanceName orcl1 -sysDBAUserName sys -sysDBAPassword oracle此时查看数据库信息：$srvctl config database -d orclDatabase unique name: orclDatabase name: Oracle user: oracleSpfile: Domain: Start options: openS

9、top options: immediateDatabase role: PRIMARYManagement policy: AUTOMATICServer pools: orclDatabase instances: his2 -原来是his1,his2Disk Groups: DATAMount point paths: Services: Type: RACDatabase is administrator managed4.4 清除rac信息清除grid/grid/gridhome/oui/bin/runInstaller -updateNodeList ORACLE_HOME=/gr

10、id/gridhome CLUSTER_NODES=his2 CRS=TRUE -silentStarting Oracle Universal Installer.Checking swap space: must be greater than 500 MB. Actual 20480 MB PassedThe inventory pointer is located at /etc/oraInst.locThe inventory is located at /grid/gridbase/oraInventoryUpdateNodeList was successful.清除oracle

11、/oracle/product/11.2.0/db_1/oui/binrunInstaller -updateNodeList ORACLE_HOME= /oracle/product/11.2.0/db_1 CLUSTER_NODES=his2Starting Oracle Universal Installer.Checking swap space: must be greater than 500 MB. Actual 20480 MB PassedThe inventory pointer is located at /etc/oraInst.locThe inventory is

12、located at /oracle/oraInventoryUpdateNodeList was successful.查看/grid/gridhome/opmen/conf/ons.config文件usesharedinstall=trueallowgroup=truelocalport=6100 # line added by Agentremoteport=6200 # line added by Agentnodes=his2:6200 # line added by Agent如果还有his1信息,手工删除4.5 清除1节点vip$srvctl remove vip -i his1

13、-vipPlease confirm that you intend to remove the VIPs his1-vip y4.6 检查当前集群信息$olsnodes -s t$crs_stat t$cluvfy stage -post nodedel -n his1 -verbosePerforming post-checks for node removal Checking CRS integrity.Clusterware version consistency passedThe Oracle Clusterware is healthy on node his2CRS inte

14、grity check passedResult: Node removal check passedPost-check for node removal was successful.4.7 重新添加1节点到集群4.7.1 添加grid软件环境检查$cluvfy comp peer -refnode his2 -n his1 -verbose$cluvfy stage -pre nodeadd -n his1 verbose在/grid/gridhome/oui/bin下：$export IGNORE_PREADDNODE_CHECKS=Y$./addNode.sh -silent CLU

15、STER_NEW_NODES=his1 CLUSTER_NEW_VIRTUAL_HOSTNAMES=his1-vipPerforming pre-checks for node addition Checking node reachability. Node reachability check passed from node his1 Checking user equivalence. User equivalence check passed for user grid Checking node connectivity. Checking hosts config file. V

16、erification of the hosts config file successful.略.WARNING:A new inventory has been created on one or more nodes in this session. However, it has not yet been registered as the central inventory of this system. To register the new inventory please run the script at /grid/gridbase/oraInventory/orainst

17、Root.sh with root privileges on nodes his1.If you do not register the inventory, you may not be able to update or patch the products you installed.The following configuration scripts need to be executed as the root user in each new cluster node. Each script in the list below is followed by a list of

18、 nodes./grid/gridbase/oraInventory/orainstRoot.sh #On nodes hxbak1/grid/gridhome/root.sh #On nodes his1To execute the configuration scripts: 1. Open a terminal window 2. Log in as root 3. Run the scripts in each cluster node The Cluster Node Addition of /grid/gridhome was successful.Please check /tm

19、p/silentInstall.log for more details.在his1以root用户执行orainstRoot.sh和root.sh两个脚本：脚本执行过程略.到这,his1上面grid已经添加完成.Asm和监听vip等都启动了.查看集群状态gridhis2 $ crs_stat -tName Type Target State Host -ora.ARCHDG.dg ora.up.type ONLINE ONLINE his1 ora.CRSDG.dg ora.up.type ONLINE ONLINE his1 ora.DATADG.dg ora.up.type ONLINE

20、ONLINE his1 ora.ER.lsnr ora.er.type ONLINE ONLINE his1 ora.N1.lsnr ora.er.type ONLINE ONLINE his2 ora.asm ora.asm.type ONLINE ONLINE his1 ora.cvu ora.cvu.type ONLINE ONLINE his2 ora.gsd ora.gsd.type OFFLINE OFFLINE ora.SM1.asm application ONLINE ONLINE his1 ora.S1.lsnr application ONLINE ONLINE his1

21、 ora.his1.gsd application OFFLINE OFFLINE ora.his1.ons application ONLINE ONLINE his1 ora.his1.vip ora.t1.type ONLINE ONLINE his1 ora.SM2.asm application ONLINE ONLINE his2 ora.S2.lsnr application ONLINE ONLINE his2 ora.his2.gsd application OFFLINE OFFLINE ora.his2.ons application ONLINE ONLINE his2

22、 ora.his2.vip ora.t1.type ONLINE ONLINE his2 work ora.rk.type ONLINE ONLINE his1 ora.oc4j ora.oc4j.type ONLINE ONLINE his2 ora.ons ora.ons.type ONLINE ONLINE his1 ora.orcl.db ora.se.type ONLINE ONLINE his2 ora.scan1.vip ora.ip.type ONLINE ONLINE his2 4.7.2 添加oracle软件在his2服务器/oui/bin/下执行：$export IGNO

23、RE_PREADDNODE_CHECKS=Y$ ./addNode.sh CLUSTER_NEW_NODES=his1Saving inventory on nodes . 100% Done.Save inventory completeWARNING:The following configuration scripts need to be executed as the root user in each new cluster node. Each script in the list below is followed by a list of nodes./oracle/prod

24、uct/11.2.0/db1/root.sh #On nodes his1To execute the configuration scripts: 1. Open a terminal window 2. Log in as root 3. Run the scripts in each cluster nodePlease check /tmp/silentInstall.log for more details.在his1节点上以root用户执行脚本Performing root user operation for Oracle 11g The following environmen

25、t variables are set as: ORACLE_OWNER= oracleEnter the full pathname of the local bin directory: /usr/local/bin: The contents of dbhome have not changed. No need to overwrite.The contents of oraenv have not changed. No need to overwrite.The contents of coraenv have not changed. No need to overwrite.E

26、ntries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Finished product-specific root actions.查看his1服务器/grid/gridhome/opmn/conf/ons.conf文件more

27、 ons.configusesharedinstall=trueallowgroup=truelocalport=6100 # line added by Agentremoteport=6200 # line added by Agentnodes=his1:6200,his2:6200 # line added by Agent如果his2服务的/grid/gridhome/opmn/conf/ons.conf没有添加his1:6200信息,手工给加上4.7.3 添加数据库实例在任意节点oracle用户执行dbca注：实际当前环境只有1个实例orcl2:his2 active添加orcl1

28、实例到此,节点恢复完成.第5章 遇到的问题在添加oracle软件的时候,因为之前没有执行清除oracle的操作,仅仅是清除了grid信息,导致集群认为his1的oracle软件已经存在,所以无法传文件.解决方案是再执行：/oracle/product/11.2.0/db_1/oui/binrunInstaller -updateNodeList ORACLE_HOME= /oracle/product/11.2.0/db_1 CLUSTER_NODES=his2然后再添加的时候报错：Error ocurred while retrieving node numbers of the exist

29、ing nodes. Please check if clusterware home is properly configured.SEVERE:Error ocurred while retrieving node numbers of the existing nodes. Please check if clusterware home is properly configured.解决办法是：copy 2 files from CRS_HOME/bin to ORACLEC_HOME/bin# cp /grid/_1/bin/# chown oracle:oinstall 1/bin/olsn*然后执行./addNode.sh -silent CLUSTER_NEW_NODES=his1 CLUSTER_NEW_VIRTUAL_HOSTNAMES=his1-vip传过去之后需要重新执行orainstRoot.sh和root.sh两个脚本.