This is a basics architecture of Oracle RAC, We will discuss about new features/additions in 11g in each sections.
- Shared Disk System
- Oracle clusterware
- Cluster interconnects
- Oracle Kernel Components
Shared Disk System
Below are the three major type of shared storage which are using in RAC
Raw volumes: A raw logical volume is an area of physical and logical disk space that is under the direct control of an application such as database or partition rather than under the direct control of the operating system or a file system. This as the only available option in 9i.
Cluster File system: This option is not widely used and here the cluster file system such as Oracle Cluster file system (OCFS) for MS Windows and Linux holding the all datafiles of RAC database
Automatic Storage Management (ASM): Oracle recommended storage option which is optimized for cluster file system for Oracle database files introduced in Oracle 10g.
Clusterware is the mandatory piece of software which is required to run Oracle RAC option,it provides the basic clustering support at OS level and enables oracle software to run in cluster mode.
Clusterware enables nodes to communicate each other and forms the cluster that make the nodes work as a single logical server.
Clusterware is managed by cluster ready services (CRS) using the Oracle Cluster registry (OCR) and voting disk
Cluster Ready Services (CRS)
Oracle clusterware uses CRS for interaction between the OS and the Database.CRS have five components - namely Process Monitor daemon (OPROCd: obsolete in 11gR2),CRS daemon(CRSd),Oralce Custer Synchronization Service daemon(OCSSd),Event Volume Manager daemon(EVMd) and the Oracle Notification Service (ONS)
CRS is installed and run from ORA_CRS_HOME in 10g and GRID_HOME in 11g
Starting and Stopping CRS
crsctl stop crs # Stops Oracle clusterware
crsctl start crs # Starts Oracle clusterware
crsctl enable crs # Enables Oracle clusterware
crsctl disable crs # Disables Oracle clusterware
Note: These commands should be used in root user (Superuser)
Oracle Cluster Synchronization ServicesOCSSd provides synchronization between the nodes. Failure of OCSSd causes the machine to reboot to avoid split-brain situation.OCSSd runs as oracle user.
Event Manager Process
This runs to daemon process EVMd. The daemon process spawns a permanent child process called evmlogger and generate the events when things happens.
Oracle Notification Services
Whenever state of cluster resource changes ONS process in each node will communicate with each other and exchange the high availability information.
Oracle Process Monitor
The oracle process monitor daemon(OPROCd) identifies the potential cluster node hang and rebooting the hang node.
Oracle Cluster Registry (OCR)
OCR uses the cluster registry to keep the configuration information. This should be a shared storage and should be accessible to all the nodes in the cluster simultaneously.This shared storage is known as Oracle cluster registry (OCR)
Oracle Logical Registry(11gR2)
Similar to OCR, introduces in 11gR2 but it only stores information about the local node. It is not shared by other nodes of cluster and used by OHASd while starting or joining a cluster.
A voting disk is a shared diks that will be accessed by all the member of the nodes in the cluster.It is used as central referece for all the nodes and keeps the heartbeat information between the nodes.If any of the node is unable to ping the voding disk,cluster immediatly recogonize the comminucation failure and envicts the node from the cluster.
It is the communication path used by the cluster for the synchronization of resources and it is also used in some cases for transfer of data from one instance to another.Typically, the interconnect is a network connections that is dedicated to the server nodes of a cluster (thus is sometimes referred as private interconnect)
Single Client Access Name (SCAN)-11gR2
SCAN is a single network name that resolves in three different IPs registered in DNS or GNS.
Prior the 11g R2 if we add or remove a node from cluster it was required to changes the connection setting in client . By configuring the connection using SCAN name this problem is eliminated.
Oracle Kernel Components
Set of additional background process in each instance is known as oracle kernel components in RAC environment.Since buffer and shared pool became global in RAC , special handling is required to manage the resources to avoid conflicts and corruption.Additional background process (for RAC) and single instance background process works together and achieved this.
Global Cache and Global Enqueue Services
In Oracle RAC each instance will have its own cache but it is required for an instance to access the data blocks currently residing in another instance cache.This management and data sharing is done by Global Cache services (GCS). Blocks other than data such as locks, enqueue details and shared across the instances are known as Global Enqueue Services (GES).
Global Resource Directory
In a cluster group, then centre repository formed by all the resources is known as Global Resource Directory (GRD).GRD is managed by GCS and GES.
Oracle RAC Background Processes
- LMS Global Cache Service Process
- LMON Global Enqueue Service Monitor
- LMD Global Enqueue service Daemon
- LCK0 Instance Enqueue Process
LMS: Global Cache Services Process
LMS- Lock Manager Server Process is used in Cache Fusion.It enebles consistent copies of blocks to be transfered from a holding instance's buffer cache to a requesting instance bufer cache with out a disk write under certian conditions.
It rollbacks any uncommitted transactions for any blocks that are being requested for a consistent read by the remote instance.
LMON:Global Enqueue Services Monitor
LMON-Lock Monitor Process is responsible to manage Global Enqueue Services (GES).It maintain consistency of GCS memory in case of any process death. LMON is also responsible for the cluster reconfiguration when an instance joins or leaves the cluster. It also check for the instance death and listens for local manages.
LMD: Global Enqueue Services Daemon
LMD-Lock Manager Daemon process manages Enqueue manager service requests for GCS. It also handles deadlock detection and remote resource requests.
LCK0:Instance Enqueue Process
LCK-Lock Process manages instance resource requests and cross-instance call operations for shared resources.
Separate process using for diagnostic purpose, PMON will start incase this process dies.DIAG process will not affect any other background process.
Reference: Oracle Real Application Cluster Handbook by K.Gopalakrishnan