Oracle 12c RAC on Oracle VirtualBox

MYVITALSOFT LESSON 3

My Oracle 12c RAC on Oracle VirtualBox

In development environments virtualization is a common practice nowadays. Oracle has now even started providing official support for Production RAC databases running on Oracle Virtualized machines. Below we look at a step by step installation process for setting up the Grid Infrastructure Software with a RAC database running on a VirtualBox using Linux Operating System.


Realted Articles
Oracle 12c Database Installation On Solaris 11 Using VirtualBox
Managing PDBs using Database Configuration Assistant
Oracle 12c Database: New Features

Download Software

Download the following Software.

Oracle Linux 6
VirtualBox for Window hosts
Oracle Database 12c Release 1 – Linux x86-64

Oracle VirtualBox Installation

Run the VirtualBox-4.2.16-86992-Win.exe and complete the VirtualBox installation.


Oracle 12c RAC on VirtualBOX

Oracle VirtualBox Setup

Create a new Virtual machine.

Name the machine and choose the Linux OS.

Oracle 12c RAC on VMware


Use an appropiate memory size.
Oracle 12c RAC Virtual


Use the default to create new hard disk.
Oracle 12c RAC on VirtualBOX


Leave the default VDI disk format.
Oracle 12c RAC on VirtualBOX 1


Use “Dynamically allocated” disk.
Oracle 12c RAC install on VMWare


Choose file location and size.
Oracle 12c RAC Installation on Linux 6 OS


Click on “Create” disk.
Oracle 12c RAC installation on VMWare


Choose hardware to use with the virtual machines.
Oracle 12c RAC on VMware 1


Confirm Network Settings for adapter 1.
Oracle 12c RAC on VirtualBOX 1


Confirm Network Settings for adapter 2.
Oracle 12c RAC Virtual


Click next to complete.
Oracle 12c RAC on VMware

Oracle Linux 6 OS Installation


Choose Enterprise 6 from the Media to install.
Oracle 12c RAC Virtual 1


Click to start.
Oracle 12c RAC on VMware 1


Leave the default to install the OS.
Oracle 12c on VMWare


Skip testing the media.
12c RAC Installation on Linux 6 OS


Click on Next.
Oracle 12c RAC installation on VMWare


Click on Next.
Oracle 12c on VMWare


Choose the language and continue.
Oracle 12c RAC install on VMWare 1


Use Basic storage.
12c RAC Quick install


Leave deafult. Ensure that the data at the location can be discarded.
Oracle 12c RAC Virtual 1


Choose the hostname.
Oracle 12c on VMWare 1

Setup Network Interface on Virtualhost

Setup eth0 and eth1.
Oracle 12c RAC installation on VMWare 2


Confirm settings for eth0.
Oracle 12c RAC Network


Confirm settings for eth0.
Setup Oracle 12c RAC Network


Confirm settings for eth0.
Oracle 12c RAC Network


Confirm settings for eth0.
Oracle 12c RAC installation on VMWare 1


Confirm settings for eth1.
12c RAC Network setup


Confirm settings for eth1.
Oracle 12c RAC install on VMWare 1


Confirm settings for eth1.
Oracle 12c on VMWare 1


Choose locale.
Oracle 12c RAC Setup


Set new Password and confirm.
Oracle 12c RAC Installation on Linux 6 OS


Choose the appropriate option.
Oracle 12c RAC Quick Install


Confirm creating disk partitions.
Oracle 12c installation on Linux 6 OS


Choose basic server.
12c RAC Quick install


Pick the appropriate choices and continue.
Oracle 12c RAC Best Install


Pick the appropriate choices and continue.
12c RAC Quick install


Pick the appropriate choices and continue.


Pick the appropriate choices and continue.


Starting installation process.
Oracle 12c RAC Quick Install


Click forward
Oracle 12c RAC Quick Install


Agree with license to continue installation
12c RAC Installation on Linux 6 OS


Register for Software updates later.


Confirm to register later
Oracle 12c RAC Installation on Linux 6 OS


Finish updates setup.
12c RAC Quick install


Click on forward.
12c RAC Installation on Linux 6 OS


Confirm your date and time.


Leave the default and click on finish
12c RAC Quick install


Initial host configuration completed.
Oracle 12c installation on Linux 6 OS

Oracle Grid infrastructure Prerequisites

Step 1:    Verify  oracle-rdbms-server-11gR2-preinstall-1.0-7.el6.x86_64 rpm on both the nodes


[[email protected] ~]# rpm -qa | grep 11g
oracle-rdbms-server-11gR2-preinstall-1.0-7.el6.x86_64


Step 2:  Adjust /etc/hosts on both the nodes as below


[[email protected] ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
###########Public ips############################################
10.0.0.121      12c-vst-rac1.vitalsofttech.com  12c-vst-rac1
10.0.0.122      12c-vst-rac2.vitalsofttech.com  12c-vst-rac2
###########Private ips############################################
192.168.0.121   12c-vst-rac1-priv.vitalsofttech.com     12c-vst-rac1-priv
192.168.0.122   12c-vst-rac2-priv.vitalsofttech.com     12c-vst-rac2-priv
###########Virtual ips############################################
10.0.0.123      12c-vst-rac1-vip.vitalsofttech.com      12c-vst-rac1-vip
10.0.0.124      12c-vst-rac2-vip.vitalsofttech.com      12c-vst-rac2-vip
###########scan ips############################################
10.0.0.125      vstracdb-scan.vitalsofttech.com  vstracdb-scan


Step 3:  On both the nodes update “/etc/security/limits.d/90-nproc.conf” and change
[[email protected] ~]# cat /etc/security/limits.d/90-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
*          -       nproc     16384
root       soft    nproc     unlimited


Step 4: Change the setting of SELinux to permissive by editing the “/etc/selinux/config” file, making sure the SELINUX flag is set as follows.
SELINUX=permissive


Step 5: If you have the Linux firewall enabled, you will need to disable or configure it
# service iptables stop
# chkconfig iptables off


Step 6: Either configure NTP, or make sure it is not configured so the Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes.

# service ntpd stop
Shutting down ntpd:                                        [  OK  ] # chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig
# rm /var/run/ntpd.pid

Note: we would be installing grid infrastructure with grid user and oracle software with oracle user


Step 7: Execute the all below steps as user root on both the nodes
Add the below at the end of file “/etc/security/limits.conf”


grid   soft   nofile   1024
grid   hard   nofile   65536
grid   soft   nproc    2047
grid   hard   nproc    16384
grid   soft   stack    10240
grid   hard   stack    32768


Append the /etc/resolv.conf file on both nodes, as below

[[email protected] ~]# cat /etc/resolv.conf
search localdomain

[[email protected] ~]# service dnsmasq restart
Starting dnsmasq:                                          [  OK  ]

Delete the existing oracle user and groups created by the OS.
userdel oracle
rm -rf /home/oracle
rm -rf /var/mail/oracle
groupdel oinstall
groupdel dba


Create new groups for grid user and oracle user

groupadd -g 1000 oinstall
groupadd -g 1010 dba
groupadd -g 1020 asmadmin
groupadd -g 1030 asmdba


Create the Grid user.

useradd -u 1100 -g oinstall -G asmadmin,asmdba grid
mkdir -p /u01/app/12.1.0/grid  ####{grid-home}
mkdir -p /u01/app/grid  ######{grid-base}
chown -R grid:oinstall /u01
passwd grid


Step 8: Logging as grid user and edit  .bash profile.
export ORACLE_HOME=/u01/app/12.1.0/grid
export ORACLE_SID=+ASM1
export PATH=$ORACLE_HOME/bin:$PATH
export ORACLE_BASE=/u01/app/grid
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
Note: use ORACLE_SID as per the node for 1st node use +ASM1 and for 2nd node use +ASM2 in .bash_profile


Create the oracle OS user.

useradd -u 1101 -g oinstall -G dba,asmdba oracle
mkdir -p /u01/app/oracle/product/12.1.0  ######(oracle home)
chown -R  oracle:oinstall /u01/app/oracle    #### (oracle base)
chmod -R 775 /u01
passwd oracle


Log in as Oracle user and edit  .bash profile.

export ORACLE_HOME=/u01/app/oracle/product/12.1.0
export ORACLE_SID=vstracdb1
export PATH=$ORACLE_HOME/bin:$PATH
export ORACLE_BASE=/u01/app/oracle
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

Note:use ORACLE_SID as per the node for 1st node use vstracdb1 and for 2nd node use vstracdb2 in .bash_profile


Step 8: Create Shared disks.


First shutdown both the nodes.


Go to virtualbox system settings>storage of node12c-vst-rac1 and add a fixed size harddisk


Oracle 12c RAC Installation


Select create new disk



Oracle 12c RAC Installation on Linux 6


Select VID and click Next


12c RAC Installation on Linux 6


Select Fixed size and click next



Oracle 12c RAC Installation on Linux 6


Click on create


Oracle 12c installation on Linux 6 OS


Click OK


12c RAC Installation on Linux 6


Click File>Virtual Media Manager and select newly created disk


12c RAC install Guide


Click modify > shareable and Click OK


12c RAC install Guide


Go to virtualbox system settings>storage of node12c-vst-rac2 and add newly created shareable hard disk



Oracle 12c RAC Installation on Linux 6


Oracle 12c RAC Installation


Step 9: Configure oracleasm on both the nodes as root user.

[[email protected] ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin

Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]:

Writing Oracle ASM library driver configuration: done

[[email protected] ~]# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm


Verify the shared disk /dev/sdb exist on both nodes using below command
[[email protected] ~]# fdisk -l /dev/sdb
Disk /dev/sdb: 4260 MB, 4260364288 bytes
255 heads, 63 sectors/track, 517 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Create a new partition in /dev/sdb using the below option (this has to be done only on one node.

[[email protected] ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xd763cd1b.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Warning: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n

Command action
e   extended
p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-517, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-517, default 517):
Using default value 517
Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.


Verify the new partition /dev/sdb1 on both the nodes using the below command.
[[email protected] ~]# fdisk -l /dev/sdb

Disk /dev/sdb: 9663 MB, 9663676416 bytes
255 heads, 63 sectors/track, 1174 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x9590bc78
Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1174     9430123+  83  Linux


Run oracleasm createdisk vstsharedisk /dev/sdb1 on node1 (12c-vst-rac1)

[[email protected] ~]# oracleasm createdisk vstsharedisk /dev/sdb1

Writing disk header: done
Instantiating disk: done

[[email protected] ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

[[email protected] ~]# oracleasm listdisks
VSTSHAREDISK


Execute the below steps on node 2 (12c-vst-rac2) to scan and list newly created oracleasm disk

[[email protected] ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin

Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done

[[email protected] ~]# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm

[[email protected] ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "VSTSHAREDISK"

[[email protected] ~]# oracleasm listdisks
VSTSHAREDISK

Note : copied the Oracle grid Infrastructure and oracle database software to / Software directory.

Grid Infrastructure Installation

Install the following package from the Oracle grid media as the root user on both the nodes.

[[email protected] rpm]#  cd /softwares/grid/rpm
[[email protected] rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm
Preparing...                ########################################### [100%] Using default group oinstall to install package
1:cvuqdisk               ########################################### [100%]/softwares/grid/rpm
login as grid user on node1 and perform the following step
[[email protected] ~]$ cd /softwares/grid/
[[email protected] grid]$ ls
install   rpm           runInstaller  stage
response  runcluvfy.sh  sshsetup      welcome.html

[[email protected] grid]$ ./runInstaller


Skip Oracle software updates.
12c RAC install


Choose to configure Standard cluster.

Oracle 12c RAC Installation on Linux 6


Choose typical installation and continue.
Oracle 12c RAC install Guide


Proceed with typical installation.
Oracle 12c on Linux 6 OS


Add/confirm the cluster nodes.
Oracle 12c installation on Linux 6


Click on next.
Oracle 12c RAC


Oracle checking SSH connectivity between all nodes.
Oracle 12c RAC Installation 1


Press ok to continue.
Oracle 12c installation on Linux 6 OS


Verify network interfaces.
12c RAC Installation on Linux 6


Provide/confirm SCAN name.
12c RAC install


Type in the passwords.
Oracle 12c RAC Setup


Provide ASM disk discovery paths.
Oracle 12c RAC install Guide


View disks available and confirm.
12c RAC install Guide


Provide/confirm inventory location.
Oracle 12c installation on Linux 6


Click on next.
Oracle 12c on Linux 6 OS


Pre-verification in progress.
Oracle 12c RAC Installation


Resolve or skip and issues discovered.
Oracle 12c RAC


Confirm and continue.
Oracle 12c RAC


Click on install.
Oracle 12c RAC Installation 1


Installation in progress.
12c RAC install


Installation almost completed. Run the root and orainsRoot scripts.
Oracle 12c on Linux 6


Run the above scripts in the mentioned order.

[[email protected] ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.

[[email protected] ~]# /u01/app/12.1.0/grid/root.sh
Performing root user operation for Oracle 12c
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME=  /u01/app/12.1.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params
2013/07/07 11:08:07 CLSRSC-363: User ignored prerequisites during installation


Final installaton steps in progress.
Oracle 12c RAC install on Laptop


Grid Infrastructure installation completed! Moving to database software installation.
Oracle 12c RAC Setup

[[email protected] ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ONLINE  ONLINE       12c-vst-rac1             STABLE ora.DATA.dg
ONLINE  ONLINE       12c-vst-rac2             STABLE ora.LISTENER.lsnr
ONLINE  ONLINE       12c-vst-rac1             STABLE
ONLINE  ONLINE       12c-vst-rac2             STABLE ora.asm
ONLINE  ONLINE       12c-vst-rac1             Started,STABLE
ONLINE  ONLINE       12c-vst-rac2             Started,STABLE ora.net1.network
ONLINE  ONLINE       12c-vst-rac1             STABLE
ONLINE  ONLINE       12c-vst-rac2             STABLE ora.ons
ONLINE  ONLINE       12c-vst-rac1             STABLE
ONLINE  ONLINE       12c-vst-rac2             STABLE

--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.12c-vst-rac1.vip
1        ONLINE  ONLINE       12c-vst-rac1             STABLE ora.12c-vst-rac2.vip
1        ONLINE  ONLINE       12c-vst-rac2             STABLE ora.LISTENER_SCAN1.lsnr
1        ONLINE  ONLINE       12c-vst-rac1             STABLE ora.cvu
1        ONLINE  ONLINE       12c-vst-rac1             STABLE ora.oc4j
1        OFFLINE OFFLINE                               STABLE ora.scan1.vip
1        ONLINE  ONLINE       12c-vst-rac1             STABLE
--------------------------------------------------------------------------------


Oracle 12c RAC Installation

Database Software Installation

Go to the runInstaller directory location.

cd /softwares/database/runInstaller


Launch runInstaller.
Oracle 12c on Linux 6


Choose appropriately and click on next.
Oracle 12c RAC Install


Confirm choice and continue.
Oracle 12c RAC Installation


Skip software updates.
Oracle 12c RAC install Guide


Choose install database software only.
Oracle 12c on Linux 6 OS


Choose RAC database installation.
Oracle 12c installation on Linux 6 OS


SSH connectivity check in progress.
Oracle 12c RAC Install


SSH Verification completed.
12c RAC Installation on Linux 6


Choose the appropriate language.
Oracle 12c RAC Installation 1


Pick the appropriate database edition.
Oracle 12c RAC Setup


Confirm Software install location.
Oracle 12c RAC install on Laptop


Verify user and groups then click on next.
Oracle 12c RAC install Guide


Prerequisite check in progress.
12c RAC install


Prerequisite check completed.
Oracle 12c RAC Install


Installation Database Software.
12c RAC Installation on Linux 6


Copy and run root.sh script as root user.
12c RAC install Guide


Database software installation completed! Moving to executing DBCA.
12c RAC Quick install


Launch the dbca binary.
Oracle 12c installation on Linux 6 OS


Provide/confirm the directory and other information.
Oracle 12c RAC Installation 1


Pre-requisite check in progress.
Oracle 12c RAC install on Laptop


Pre-requisite check completed.
Oracle 12c RAC Setup


Datababase creation summary. Click on finish.
Oracle 12c RAC install Guide


Database creation in progress.
12c RAC install Guide


Database creation in progress.
Oracle 12c RAC Install


Note the database details.
12c RAC Installation on Linux 6


Installation completed!


Read more on – What is Oracle 12c?