Skip to main content

Shell Script for Bringing RAC(CRS)/Standalone(HAS) services Down/Up




SHELL SCRIPT for DB Server UNIX PATCHING


Shell script for bringing RAC /Standalone(HAS) services down/up with one script. These scripts will perform following options.

1. Creates Blackout node level in all Nodes
2. Takes current service status backup and disables the crontab
3. Brings down the Database services and Databases
4. Brings down the Cluster services
5. Disables the CRS/HAS service

stop_RAC_services.sh  : Please modify the agent home location (highlighted on RED color)

#!/bin/sh
## Script for Unix patching
## Author : Srinivas P

if [[ $0 != "/"* ]]; then
LOGDIR=`pwd`"/"`dirname $0`
else
LOGDIR=`dirname $0`
fi
cd $LOGDIR
LOGDIR=`pwd`

ASM="$LOGDIR/log/sidname"
LST="$LOGDIR/log/listhome"
DBN="$LOGDIR/log/dbname"
DBS="$LOGDIR/log/dbserv"
CLU="$LOGDIR/log/clus"
SERVERS="$LOGDIR/log/serv"


if [ ! -d "$LOGDIR/log" ]
then
mkdir $LOGDIR/log
touch $SERVERS
else
mv $LOGDIR/log $LOGDIR/log_bkp_$(date +"%m-%d-%y")
mkdir $LOGDIR/log
touch $SERVERS
fi

function dbn
{
sid8=`ps -ef | grep pmon | grep ASM |awk '{print $8 }'|cut -c 10-18`
export ORACLE_SID=$sid8
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
if [ -f $SERVERS ]
then
rm -f $SERVERS
olsnodes > $SERVERS
if [ -s $SERVERS ]
then
rm -rf $LOGDIR/log/dblist
ps -ef | grep ora_pmon | grep -v grep  | grep -v ASM | grep -v APX | grep -v MGMTDB | awk '{ print substr($8,10)}' | rev | cut -c 2- |rev  > $LOGDIR/log/dblist
echo "C" > $CLU
CLU=C
else
rm -rf $LOGDIR/log/dblist
ps -ef | grep ora_pmon | grep -v grep  | grep -v ASM | grep -v APX | grep -v MGMTDB | awk '{ print substr($8,10)}' > $LOGDIR/log/dblist
hostname > $SERVERS
echo "S" > $CLU
CLU=S
fi
else
echo "file not exists "
fi
}


function blackout {
export AGENT_HOME=/u01/app/oracle/product/agent/agent_inst
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
ssh $SERVER "$AGENT_HOME/bin/emctl start blackout UnixPatch -nodelevel"
done
}

function cluster {
if [ ${CLU} == C ]
then
sid=`ps -ef | grep pmon | grep ASM |awk '{print $8 }'|cut -c 10-18`
echo "$sid" > "${ASM}"
export ORACLE_SID=$sid
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
sudo $ORACLE_HOME/bin/crsctl stop cluster -all
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
ssh -t $SERVER ". ~/.profile > /dev/null;`cat /etc/oratab | grep ASM | awk -F ":" '{print $2'} | sort -n -u` sudo $ORACLE_HOME/bin/crsctl stop crs"
ssh -t $SERVER ". ~/.profile > /dev/null;`cat /etc/oratab | grep ASM | awk -F ":" '{print $2'} | sort -n -u` sudo $ORACLE_HOME/bin/crsctl disable crs"
done
unset ORAENV_ASK
else if [ ${CLU} == S ]
then
sid=`ps -ef | grep pmon | grep ASM |awk '{print $8 }'|cut -c 10-18`
echo "$sid" > "${ASM}"
export ORACLE_SID=$sid
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
$ORACLE_HOME/bin/crsctl stop has
$ORACLE_HOME/bin/crsctl disable has
unset ORAENV_ASK
else
echo " Failed to stop Cluster services"
fi
fi
}


function backup
{
sid3=`ps -ef | grep pmon | grep ASM |awk '{print $8 }'|cut -c 10-18`
export ORACLE_SID=$sid3
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
$ORACLE_HOME/bin/crsctl stat res -t > $LOGDIR/log/crsctl_bkp_$(date +"%m-%d-%y")
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
ssh $SERVER ". ~/.profile > /dev/null; cat /etc/oratab " > $LOGDIR/log/oratab_${SERVER}_bkp
ssh $SERVER ". ~/.profile > /dev/null; df -h " > $LOGDIR/log/mountpoint_${SERVER}_bkp
ssh $SERVER ". ~/.profile > /dev/null; crontab -l " > $LOGDIR/log/crontab_${SERVER}_bkp
ssh $SERVER ". ~/.profile > /dev/null; crontab -r"
done
unset ORAENV_ASK
}

function dbstop
{
if [ ${CLU} == C ]
then
ps -ef | grep pmon | grep -v grep | grep -v ASM  |grep -v APX |grep -v MGMTDB | awk '{print $8}'|cut -c 10-18 > ${DBN}
for sid4 in `cat ${DBN}| tr -s ',' ' '| sed 's/.$//'`
do
sid=`ps -ef | grep pmon | grep ${sid4} |awk '{print $8 }'|cut -c 10-18`
export ORACLE_SID=$sid
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
echo "$ORACLE_HOME/bin/srvctl stop database -d $sid4"
$ORACLE_HOME/bin/srvctl stop database -d $sid4
unset ORAENV_ASK
done
else if [ ${CLU} == S ]
then
ps -ef | grep pmon | grep -v grep | grep -v ASM | awk '{print $8 }'|cut -c 10-18 > ${DBN}
for sid4 in `cat ${DBN}| tr -s ',' ' '`
do
sid5=`ps -ef | grep pmon | grep ${sid4} |awk '{print $8 }'|cut -c 10-18`
export ORACLE_SID=$sid5
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
echo "$ORACLE_HOME/bin/srvctl stop database -d $sid4"
$ORACLE_HOME/bin/srvctl stop database -d $sid4
unset ORAENV_ASK
done
fi
fi
}

function db_stop_service
{
if [ ${CLU} == C ]
then
ps -ef | grep pmon | grep -v grep | grep -v ASM  |grep -v APX |grep -v MGMTDB | awk '{print $8}'|cut -c 10-18 > ${DBS}
for sid7 in `cat ${DBS}| tr -s ',' ' '| sed 's/.$//'`
do
sid6=`ps -ef | grep pmon | grep ${sid7} |awk '{print $8 }'|cut -c 10-18`
export ORACLE_SID=$sid6
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
file_output="${LOGDIR}/log/srv_info_${sid7}"
$ORACLE_HOME/bin/srvctl status service -d $sid7 | grep -i "is running" | awk '{print $2}' > ${file_output}
for srv in `cat ${file_output} | tr -s ',' ' '`
do
echo "$ORACLE_HOME/bin/srvctl stop service -s $srv -d $sid7"
$ORACLE_HOME/bin/srvctl stop service -s $srv -d $sid7
done
done
else if [ ${CLU} == S ]
then
ps -ef | grep pmon | grep -v grep | grep -v ASM  |grep -v APX |grep -v MGMTDB | awk '{print $8}'|cut -c 10-18 > ${DBS}
for sid7 in `cat ${DBS}| tr -s ',' ' '`
do
sid6=`ps -ef | grep pmon | grep ${sid7} |awk '{print $8 }'|cut -c 10-18`
export ORACLE_SID=$sid6
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
file_output="${LOGDIR}/log/srv_info_${sid7}"
$ORACLE_HOME/bin/srvctl status service -d $sid7 | grep -i "is running" | awk '{print $2}' > ${file_output}
for srv in `cat ${file_output} | tr -s ',' ' '`
do
echo "$ORACLE_HOME/bin/srvctl stop service -s $srv -d $sid7"
$ORACLE_HOME/bin/srvctl stop service -s $srv -d $sid7
done
done
fi
fi
}


function main {
echo " You are going to shutdown the database and cluster on `hostname`"
echo "`ps -ef | grep pmon`"
echo " ################################################################"
read -p " Are you sure want to continue (y/n)" prompt
if [ $prompt = y ]
then
blackout
sleep 10
echo "Please check if blackout was posted or not Either from OEM or command line from different session to avoid critical alerts example: emctl status blackout"
echo "################################################################"
read -p "Are you sure want to continue (y/n)" prmpt
if [ $prmpt = y ]
then
echo " Backup of crontab, oratab, service status etc"
backup
sleep 10
echo " DB services are getting stopped"
db_stop_service
sleep 10
echo " Databases are getting stopped"
dbstop
sleep 10
echo " Cluster is getting stopped"
cluster
echo "#################################################################"
echo " Database services has been stopped, please proceed with patching"
echo "#################################################################"
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
echo "$SERVER"
ssh $SERVER "echo "`ps -ef | grep pmon`""
ssh $SERVER "echo "`ps -ef | grep d.bin`""
done
else
echo "blackout is not posted correctly hence exiting the script"
fi
else
echo " Script ended by not shutting down the database and cluster"
fi

}

dbn
if [ -f $SERVERS ]
then
main |& tee ${LOGDIR}/log/services_down.log
else
echo " Please check if you have created servers file and added node names with domain names example: pc1uusbkrac01.usbk.prod"
fi


====================================================================

Below script is useful for starting up the database services. This script will perform following options.

1. Enables the CRS/HAS
2. Starting Cluster services
3. Starting Database and database services in preferred nodes.
4. enables the crontab jobs
5. Removes the blackout

Also, this script has two options for better use. we can skip crontab and blackout steps by passing "-s" option then perform only these two steps by passing "-o" option.

start_RAC_Dbservices.sh

#!/bin/sh
## Script for Unix patching
## Author : Srinivas P

if [[ $0 != "/"* ]]; then
LOGDIR=`pwd`"/"`dirname $0`
else
LOGDIR=`dirname $0`
fi
cd $LOGDIR
LOGDIR=`pwd`

ASM="$LOGDIR/log/sidname"
DBN="$LOGDIR/log/dbname"
DBS="$LOGDIR/log/dbserv"
SERVERS="$LOGDIR/log/serv"
CLU=`cat $LOGDIR/log/clus`

DEF="N"
BKO="${DEF}"
BLO="${DEF}"

while getopts "sS:oO" opt ;
do
echo "$opt" $OPTARG
case $opt in
 s) BKO="S" ;;
 S) BKO="S" ;;
 O) BLO="O" ;;
 o) BLO="O" ;;
 esac
done


function blackout {
export AGENT_HOME=/u01/app/oracle/product/agent/agent_inst
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
ssh $SERVER "$AGENT_HOME/bin/emctl start agent"
ssh $SERVER "$AGENT_HOME/bin/emctl stop blackout UnixPatch"
sleep 10
ssh $SERVER "$AGENT_HOME/bin/emctl upload"
done
}

function cluster {
if [ ${CLU} == C ]
then
sid=$(awk 'NR==1 {print;exit}' ${ASM})
export ORACLE_SID=$sid
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
ssh -t $SERVER ". ~/.profile > /dev/null;`cat /etc/oratab | grep ASM | awk -F ":" '{print $2'} | sort -n -u` sudo $ORACLE_HOME/bin/crsctl enable crs"
ssh -t $SERVER ". ~/.profile > /dev/null;`cat /etc/oratab | grep ASM | awk -F ":" '{print $2'} | sort -n -u` sudo $ORACLE_HOME/bin/crsctl start crs"
done
rm -rf "${ASM}"
unset ORAENV_ASK
else if [ ${CLU} == S ]
then
sid=$(awk 'NR==1 {print;exit}' ${ASM})
export ORACLE_SID=$sid
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
$ORACLE_HOME/bin/crsctl enable has
$ORACLE_HOME/bin/crsctl start has
rm -rf "${ASM}"
unset ORAENV_ASK
fi
fi
}

function dbstart
{
if [ ${CLU} == C ]
then
for sid4 in `cat ${DBN}| tr -s ',' ' '| sed 's/.$//'`
do
sid=`cat ${DBN} | grep ${sid4}`
export ORACLE_SID=$sid
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
echo "$ORACLE_HOME/bin/srvctl start database -d $sid4"
$ORACLE_HOME/bin/srvctl start database -d $sid4
done
unset ORAENV_ASK
else if [ ${CLU} == S ]
then
for sid4 in `cat ${DBN} | tr -s ',' ' '`
do
export ORACLE_SID=$sid4
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
echo "/u01/app/12.1.0.2/grid/bin/srvctl start database -d $sid4"
/u01/app/12.1.0.2/grid/bin/srvctl start database -d $sid4
unset ORAENV_ASK
done
rm -rf ${DBN}
fi
fi
}

function dbservices
{
if [ ${CLU} == C ]
then
for sid4 in `cat ${DBS}| tr -s ',' ' '| sed 's/.$//'`
do
sid=`cat ${DBS} | grep ${sid4}`
export ORACLE_SID=$sid
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
$ORACLE_HOME/bin/srvctl status service -d $sid4 | grep -i "is running" | awk '{print $2}' > srv_text
for srv in `cat srv_text | tr -s ',' ' '`
do
$ORACLE_HOME/bin/srvctl stop service -s $srv -d $sid4
done
sleep 20
file_output="${LOGDIR}/log/srv_info_${sid4}"
for srv in `cat ${file_output} | tr -s ',' ' '`
do
echo "$ORACLE_HOME/bin/srvctl start service -s $srv -d $sid4"
$ORACLE_HOME/bin/srvctl start service -s $srv -d $sid4
done
unset ORAENV_ASK
done
rm -rf ${DBS}
else if [ ${CLU} == S ]
then
for sid4 in `cat ${DBS}| tr -s ',' ' '`
do
sid=`cat ${DBS} | grep ${sid4}`
export ORACLE_SID=$sid
export ORAENV_ASK=NO
export PATH=$PATH:/usr/local/bin/:/usr/loca/sbin
. oraenv
$ORACLE_HOME/bin/srvctl status service -d $sid4 | grep -i "is running" | awk '{print $2}' > srv_text
for srv in `cat srv_text | tr -s ',' ' '`
do
echo "$ORACLE_HOME/bin/srvctl stop service -s $srv -d $sid4"
$ORACLE_HOME/bin/srvctl stop service -s $srv -d $sid4
done
sleep 60
file_output="${LOGDIR}/log/srv_info_${sid4}"
for srv in `cat ${file_output} | tr -s ',' ' '`
do
echo "$ORACLE_HOME/bin/srvctl start service -s $srv -d $sid4"
$ORACLE_HOME/bin/srvctl start service -s $srv -d $sid4
done
unset ORAENV_ASK
done
rm -rf ${DBS}
fi
fi
}


function main {
echo " You are going to start cluster and database on `hostname`"
echo "###########################################################"
read -p "Are you sure want to continue (y/n)" prompt
if [ $prompt = y ]
then
echo " Cluster is starting"
cluster
sleep 360
CNT=`ps -ef | grep pmon | grep ASM |wc -l`
if [ ${CNT} -gt 0 ]
then
echo " Databases are getting started"
dbstart
sleep 60
echo " Database services are starting in preffered nodes"
dbservices
else
sleep 380
echo " Databases are getting started"
dbstart
sleep 60
echo " Database services are starting in preffered nodes"
dbservices
fi
sleep 60
if [ $BKO = ${DEF}  ]
then
blackout
echo "##########################################################################"
echo " Database services has been started, verify if the database are up or not"
echo "##########################################################################"
echo "Copying crontab scripts"
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
ssh $SERVER "crontab " < $LOGDIR/log/crontab_${SERVER}_bkp
done
else
echo "Blackout has ben skipped as requested"
echo "#####################################"
fi
echo "verifying database services are started or not "
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
ssh $SERVER "echo "`ps -ef | grep pmon`""
ssh $SERVER "echo "`ps -ef | grep d.bin`""
done
else
echo " Script ended by not performing anything "
fi
}

if [ -f $SERVERS ]
then
if  [ $BLO != "${DEF}" ]
then
blackout
echo "Copying crontab scripts"
for SERVER in `cat $SERVERS | tr -s ',' ' ' `
do
ssh $SERVER "crontab " < $LOGDIR/log/crontab_${SERVER}_bkp
done
else
main |& tee ${LOGDIR}/log/services_up.log
fi
else
echo " Servers file doesn't exist. Kindly verify "
fi


======================================================================




Comments

Popular posts from this blog

How to unmount and mount ACFS FILESYSTEM

How to unmount and mount ACFS FileSystem in Linux Unmounting ACFS File Systems ACFS file systems can be used by Oracle RAC for hosting software files for the database. It can also be used as a general purpose file system for non-database files. The ACFS file system is managed and administered by the Oracle Grid Infrastructure, So ACFS file systems will be impacted when shutting down the GI stack for patching GI homes. Shut down the processes (including non-Oracle processes) that use the files on ACFS, then unmount the ACFS file system. If the ACFS file system is used by Oracle database software, then perform Steps 1 and 2. 1.      Execute the following command to find the names of the CRS managed ACFS file system resource.  # crsctl stat res -w "TYPE = ora.acfs.type" -p | grep VOLUME 2.      Execute the following command to stop the CRS managed ACFS file system resource with the resource name found from Step 1. ...

How to perform GRID/RDBMS patching of DB systems deployed in OCI using dbcli command line utility

This procedure covers OS, Grid and RDBMS home patching for Oracle Cloud Infrastructure (OCI) DB systems using the dbcli utility. Run every step as root . Advantages of dbcli patching Simple command line, with no manual patch download or copy to the nodes. Patches are downloaded automatically if the service gateway can reach Oracle Object Storage and the YUM repos. Applied in a rolling fashion, so no downtime is needed. Disadvantages You can't choose the patch version; the latest available in OCI is applied. Rollback isn't possible from the console, and dbcli support is limited (see the note at the end). Steps 1. Backup Make sure a recent full backup exists before patching. 2. Blackout Create a blackout for both nodes (console or emctl ) to avoid unnecessary alerts. 3. Check patch availability # dbcli describe-component 4. Pre-checks Grid home: # dbcli update-server -p # dbcli describe-job -i <jobId> RDBMS home: # dbcli list-dbhomes...

To fix huge lag between primary and DR using 12c feature: 12c Recover from service

Scope If transport or apply lag is huge difference, then we can fix using 12c new feature “Recover from Service”. Prerequisites : Oracle Net connectivity is established between the physical standby database and the primary database. You can do this by adding an entry corresponding to the primary database in the tnsnames.ora file of the physical standby database. The password files on the primary database and the physical standby database are the same. The COMPATIBLE parameter in the initialization parameter file of the primary database and physical standby database is set to 12.0 or higher. Start RMAN and connect as target to the physical standby database. Detailed Steps: Step 1: Stop the MRP process in standby database. If dg broker: Edit database <dbname> set state=’APPLY-OFF’; Step 2:   Shut down the standby database and mount the same. Shut immediate;   (srvctl stop database –d <dbname>)  ...