Recent from talks
Nothing was collected or created yet.
Oracle RAC
View on WikipediaOracle Real Application Clusters (RAC) is an option[1] for the Oracle Database software produced by Oracle Corporation and introduced in 2001 with Oracle9i. It provides software for clustering and high availability in Oracle database environments. Oracle Corporation includes RAC with the Enterprise Edition, provided the nodes are clustered using Oracle Clusterware.[2]
Functionality
[edit]Oracle RAC allows multiple computers to run Oracle RDBMS software simultaneously while accessing a single database, thus providing clustering.
In a non-RAC Oracle database, a single instance accesses a single database. The database consists of a collection of data files, control files, and redo logs located on disk. The instance comprises the collection of Oracle-related memory and background processes that run on a computer system.
In an Oracle RAC environment, 2 or more instances concurrently access a single database. This allows an application or user to connect to either computer and have access to a single coordinated set of data. The instances are connected with each other through an "Interconnect" which enables all the instances to be in sync in accessing the data.
Aims
[edit]The main aim of Oracle RAC is to implement a clustered database to provide performance, scalability and resilience & high availability of data at instance level.
Implementation
[edit]Oracle RAC depends on the infrastructure component Oracle Clusterware to coordinate multiple servers and their sharing of data storage.[3]
The FAN (Fast Application Notification) technology detects down-states.[4]
RAC administrators can use the srvctl tool to manage RAC configurations,[5]
Cache Fusion
[edit]Prior to Oracle 9, network-clustered Oracle databases used a storage device as the data-transfer medium (meaning that one node would write a data block to disk and another node would read that data from the same disk), which had the inherent disadvantage of lackluster performance. Oracle 9i addressed this issue: RAC uses a dedicated network connection for communications internal to the cluster.
Since all computers/instances in a RAC access the same database, the overall system must guarantee the coordination of data changes on different computers such that whenever a computer queries data, it receives the current version — even if another computer recently modified that data. Oracle RAC refers to this functionality as Cache Fusion. Cache Fusion involves the ability of Oracle RAC to "fuse" the in-memory data cached physically separately on each computer into a single, global cache.
Networking
[edit]The Oracle Grid Naming Service (GNS) handles name resolution in the cluster registry.[6]
Diagnostics
[edit]The Trace File Analyzer (TFA) aids in collecting RAC diagnostic data.[7]
Versions
[edit]Evolution
[edit]Relative to the single-instance Oracle database, Oracle RAC adds additional complexity. While database automation makes sense for single-instance databases, it becomes even more necessary for clustered databases because of their increased complexity.
Oracle Real Application Clusters (RAC), introduced with Oracle 9i in 2001, supersedes the Oracle Parallel Server (OPS) database option. Whereas Oracle9i required an external clusterware (known as vendor clusterware like TruCluster Veritas Cluster Server or Sun Cluster) for most of the Unix flavors (except for Linux and Windows where Oracle provided free clusterware called Cluster Ready Services or CRS), as of Oracle 10g, Oracle's clusterware product was available for all operating systems. With the release of Oracle Database 10g Release 2 (10.2), Cluster Ready Services was renamed to Oracle Clusterware. When using Oracle 10g or higher, Oracle Clusterware is the only clusterware that you need for most platforms on which Oracle RAC operates (except for Tru cluster, in which case you need vendor clusterware). You can still use clusterware from other vendors, if the clusterware is certified for Oracle RAC.
In RAC, the write-transaction must take ownership of the relevant area of the database: typically, this involves a request across the cluster interconnection (local IP network) to transfer the data-block ownership from another node to the one wishing to do the write. This takes a relatively long time (from a few to tens of milliseconds) compared to single database-node using in-memory operations. For multiple types of applications, the time spent coordinating block access across systems is low relative to the multiple operations on the system, and RAC will scale comparably to a single system.[citation needed] Moreover, high read-transactional databases (such as data-warehousing applications) work very well under RAC, as no need for ownership-transfer exists. (Oracle 11g has made a number of enhancements in this area and performs a lot better than earlier versions for read-only workloads.[citation needed])
The overhead on the resource mastering (or ownership-transfer) is minimal for fewer than three nodes, as the request for any resource in the cluster can be obtained in a maximum of three hops (owner-master-requestor).[citation needed] This makes Oracle RAC horizontally scalable with a number of nodes. Application vendors (such as SAP) use Oracle RAC to demonstrate the scalability of their application. Most of the biggest OLTP benchmarks are on Oracle RAC. Oracle RAC 11g supports up to 100 nodes.[10]
For some[which?] applications, RAC may require careful application partitioning to enhance performance. An application that scales linearly on an SMP machine may scale linearly under RAC. However, if the application cannot scale linearly on SMP, it will not scale when ported to RAC. In short, the application scalability is based on how well the application scales in a single instance.
Competitive context
[edit]Shared-nothing and shared-everything architectures each have advantages over the other. DBMS vendors and industry analysts regularly debate the matter; for example, Microsoft touts a comparison of its SQL Server 2005 with Oracle 10g RAC.[11]
Oracle Corporation offered a Shared Nothing architecture RDBMS with the advent of the IBM SP and SP2 with the release of 7.x MPP editions, in which virtual shared drives (VSD) were used to create a Shared Everything implementation on a Shared Nothing architecture.
Shared-Everything
[edit]Shared-everything architectures share both data on disk and data in memory between nodes in the cluster. This is in contrast to "shared-nothing" architectures that share none of them.
Some commercially available databases offer a "shared-everything" architecture. IBM Db2 for z/OS (the IBM mainframe operating-system) has provided a high-performance data-sharing option since the mid-1990s when IBM released its mainframe hardware and software-clustering infrastructure. In late 2009, IBM announced DB2 pureScale, a shared-disk clustering scheme for DB2 9.8 on AIX that mimics the parallel sysplex implementation behind Db2 data sharing on the mainframe.
In February 2008, Sybase released its Adaptive Server Enterprise, Cluster Edition. It resembles Oracle RAC in its shared-everything design.[12]
Although technically not shared-everything, Sybase also provides a column-based relational database focused on analytic and datawarehouse applications called Sybase IQ that can be configured to run in a shared disk mode.
Cloud Native Databases, such as Amazon Aurora and POLARDB of Alibaba Cloud, are implemented with "shared-everything" architecture on top of cloud-based distributed file system.[13][14]
Shared-nothing
[edit]Shared-nothing architectures share neither the data on disk nor the data in memory between nodes in the cluster. This is in contrast to "shared-everything" architectures, which share both.
Competitive products offering shared-nothing architectures include:
- EDB Postgres Distributed available for PostgreSQL, EDB Postgres Extended Server and EDB Postgres Advanced Server (which provides native compatibility with Oracle)
- MySQL Cluster (Oracle Corporation has owned MySQL since 2009)[15]
- Clustrix
- HP NonStop
- IBM InfoSphere Warehouse editions that include the Database Partitioning Feature (formerly known as DB2 Extended Enterprise Edition)
- MarkLogic
- Greenplum
- Oracle NoSQL Database
- Paraccel
- Netezza (aka. Netezza Performance Server)
- Teradata
- Vertica
- Apache Cassandra, wide column store NoSQL database.
- Apache HBase
- MongoDB, document-oriented database.
- Couchbase Server
- Riak
- SAP HANA
- CUBRID
See also
[edit]References
[edit]- ^ Options and Packs
- ^ Oracle Database Editions
- ^ Introduction to Oracle Real Application Clusters
- ^
Mensah, Kuassi (2006). Oracle database programming using Java and Web services. Digital Press. p. 400; 1087. ISBN 978-1-55558-329-3. Retrieved 2011-09-11.
The Fast Application Notification (FAN) mechanism [...] allows the rapid detection of "
Instance DOWN" or "Node DOWNevents [...] - ^
Stoever, Edward (2006). Personal Oracle RAC Clusters: Create Oracle 10g Grid Computing At Home. Oracle In-focus Series. Rampant TechPress. p. 119. ISBN 9780976157380. Retrieved 2013-05-30.
An RAC database configuration requires extra tools to manage the software and its instances. One such tool is srvctl, used to startup, shutdown and check the status [of] a RAC database.
- ^
Prusinski, Ben; Hussain, Syed Jaffer (23 May 2011). Oracle 11g R1/R2 Real Application Clusters Essentials. Birmingham: Packt Publishing Ltd (published 2011). ISBN 9781849682671. Retrieved 2018-03-23.
Oracle 11g R2 RAC introduced several new clusterware background processes. [...] The Oracle Grid Naming Service (GNS) functions as a gateway between the cluster mDNS and external DNS servers. The GNS process performs the name resolution within Oracle Cluster registry architecture for Oracle 11g RAC.
- ^
Farooq, Tariq; Kim, Charles; Vengurlekar, Nitin; Avantsa, Sridhar; Harrison, Guy; Hussain, Syed Jaffar (12 June 2015). "Troubleshooting and Tuning RAC". Oracle Exadata Expert's Handbook. Addison-Wesley Professional (published 2015). ISBN 9780133780987. Retrieved 2017-06-29.
Released with v11.2.0.4, the Trace File Analyzer (TFA) Collector utility is the new all-encompassing utility that simplifies collection of RAC diagnostic information.
- ^
"Oracle 12c RAC: New Features". Find White Papers. 2015-07-24. Retrieved 2015-07-24.
From among the 500+ New Features released with Oracle 12c Database, a number of very useful features are Oracle RAC specific. View the top 12c RAC New Features including Oracle ASM Flex, ASM Disk Scrubbing, faster Disk Resync Checkpoint, higher Resync Power limit and more.
- ^ "Oracle Real Application Clusters One Node: Better Virtualization for Databases". Find White Papers. 2009-12-09. Archived from the original on 2010-02-04. Retrieved 2010-04-19.
Oracle RAC One Node provides: . Always on single-instance database services . Better consolidation for database servers . Enhanced server virtualization . [,,,] should the need arise, upgrade to a full multi-node Oracle RAC database without downtime or disruption. [...] Oracle Real Application Clusters (RAC) One Node is a new option to Oracle Database 11g Release 2 Enterprise Edition. It provides enhanced high availability for singleinstance databases,
- ^ "clustering" (PDF). Oracle.com. Retrieved 2012-11-07.
- ^ Thomas, Bryan (2006-05-30). "Solutions for Highly Scalable Database Applications: An analysis of architectures and technologies" (PDF). Microsoft. Retrieved 2007-09-09.
- ^ "Sybase.com". Sybase.com. Retrieved 2012-11-07.
- ^ "Amazon Aurora storage and reliability - Amazon Aurora".
- ^ "PolarFS: An Ultra-low Latency and Failure Resilient Distributed File System for Shared Storage Cloud Database". ACM DIGITAL LIBRARY.
- ^ "Oracle buys Finnish open-source developer". InfoWorld. October 7, 2005. "Oracle Buys SUN; MySQL is Forked". Linux Magazine. April 20, 2009.
External links
[edit]Oracle RAC
View on GrokipediaIntroduction
Definition and Core Concepts
Oracle Real Application Clusters (RAC) is Oracle's shared-everything clustering solution for the Oracle Database, enabling multiple server instances to operate against a single physical database for enhanced scalability and availability.[3] Introduced in 2001 with Oracle Database 9i Release 1, RAC replaced the prior Oracle Parallel Server (OPS), which depended on disk-based locking mechanisms that limited performance in multi-node environments.[4] This architecture allows a cluster of independent servers, or nodes, to function as a unified system, sharing access to all database resources without partitioning data across nodes.[5] The core structure of Oracle RAC centers on shared storage for critical database components, including data files, control files, and redo logs, which are accessible to every node via cluster-aware file systems or storage area networks.[3] Each node runs its own database instance, complete with dedicated memory structures such as the System Global Area (SGA) for caching data blocks and the Program Global Area (PGA) for session-specific operations, along with background processes for local management.[3] Instances coordinate and synchronize through a private, high-speed interconnect network, which facilitates communication for resource allocation and data consistency.[3] In contrast to a single-instance Oracle Database, where a solitary instance handles all database operations, RAC supports active-active scaling across nodes, distributing workload dynamically to improve throughput and resilience without necessitating application code changes.[3] This setup eliminates single points of failure, as the database remains operational on surviving nodes if one fails, with synchronization primarily managed via the Cache Fusion protocol for efficient block transfers between caches.[4]Primary Objectives and Benefits
Oracle Real Application Clusters (RAC) primarily aims to deliver high availability by tolerating node failures without interrupting database operations, enabling horizontal scalability through the addition of cluster nodes, and enhancing performance via parallel query processing across multiple instances.[6] This architecture allows organizations to maintain continuous access to data even during hardware or software failures, supporting mission-critical applications that require uninterrupted service.[2] Key benefits include fault isolation, where a failure on one node does not impact others, ensuring that workloads continue seamlessly on surviving instances; load balancing across nodes to optimize resource utilization and prevent bottlenecks; and integration with Oracle Data Guard for robust disaster recovery, allowing rapid failover to a standby site while maintaining data consistency.[6] These features leverage shared storage for concurrent data access and Cache Fusion for low-latency block transfers between nodes, further minimizing disruptions.[2] In modern versions, such as Oracle Database 23ai (released 2024), Oracle RAC supports clusters with over 100 nodes, achieving near-linear scalability without application modifications, and reduces downtime to under 15 seconds through fast failover mechanisms.[6][7] It is particularly suited for online transaction processing (OLTP) systems demanding 24/7 uptime, such as banking and e-commerce platforms, as well as large-scale analytics workloads that benefit from parallel execution without data partitioning.[2]Prerequisites
Hardware and Software Requirements
Oracle Real Application Clusters (RAC) requires robust hardware configurations to ensure high availability and scalability across multiple nodes. Each node must utilize symmetric multiprocessing (SMP) servers or blade systems capable of supporting the cluster's workload, with a minimum of 8 GB RAM for Oracle Grid Infrastructure installations and at least 1 GB (recommended 2 GB or more) for the Oracle Database software.[8] Shared storage is essential, typically implemented via Storage Area Network (SAN) or Network Attached Storage (NAS) with multipath I/O for redundancy and performance, where Oracle Automatic Storage Management (ASM) is recommended for managing database files, control files, SPFILEs, redo logs, and recovery files.[9] Networking demands at least one public network interface card (NIC) per node for client access and a dedicated private interconnect using high-speed Ethernet, with a minimum of 1 Gbps and recommendations for 10 Gbps or faster to handle Cache Fusion traffic efficiently.[10] On the software side, Oracle RAC necessitates the Enterprise Edition of Oracle Database with the RAC option licensed, alongside Oracle Grid Infrastructure for cluster management, which must be installed in a separate Oracle home from the database software. For Oracle Database 21c and later (including the current 23ai release as of 2025), the multitenant architecture with a container database (CDB) is mandatory.[1][9] Compatible operating systems include certified versions of Linux (such as Oracle Linux 8 and later, Red Hat Enterprise Linux, and SUSE Linux Enterprise Server), Windows Server (e.g., 2019, 2022, 2025), and Oracle Solaris on SPARC or x86-64 platforms, ensuring identical OS configurations across all nodes to avoid compatibility issues.[11][12][13] All hardware and software must comply with Oracle's certification matrices to guarantee support and interoperability. The Oracle Hardware Compatibility List (via My Oracle Support) verifies compatible servers, storage, and network adapters from vendors like IBM, HP, and Oracle/Sun, while the software certification matrix outlines supported OS patches and versions.[11] Virtualization environments are supported, including VMware vSphere, Oracle VM, KVM, and Microsoft Hyper-V, allowing RAC deployment on certified virtual machines with performance considerations for shared resources.[14] The minimum cluster configuration consists of 2 nodes for basic redundancy, with scalability extending to 16 or more physical nodes depending on the storage protocol—such as up to 30 nodes with iSCSI over a private Gigabit Ethernet network—though practical limits are influenced by interconnect bandwidth and storage redundancy.[13][11]Cluster Setup Essentials
Setting up an Oracle Real Application Clusters (RAC) environment requires meticulous planning to ensure high availability, scalability, and performance. Initial planning steps involve conducting a capacity assessment to evaluate workload requirements, including input/output operations per second (IOPS) and throughput needs for the shared storage subsystem, which directly impacts database performance under concurrent access from multiple nodes.[15] Redundancy must be incorporated into the shared storage design, typically using RAID 1+0 configurations to provide both mirroring for fault tolerance and striping for balanced load distribution across disks.[16] Additionally, integrating a backup strategy from the outset is essential, leveraging tools like Recovery Manager (RMAN) to support consistent backups of the shared database while accounting for cluster-wide synchronization.[16] Storage configuration in Oracle RAC centers on Automatic Storage Management (ASM), which simplifies the management of shared disks by creating disk groups that span multiple nodes for pooled storage resources. ASM disk groups are used to store database files, redo logs, and control files, ensuring uniform access and automatic load balancing. Critical components include voting disks, which maintain cluster membership by recording node status—Oracle recommends configuring 3 to 5 voting disks in an ASM disk group with normal or high redundancy to tolerate failures without quorum loss. The Oracle Cluster Registry (OCR) stores cluster configuration data and must be placed in a separate ASM disk group or shared file system, with redundancy provided through mirroring to prevent single points of failure. Note that starting with Oracle Database 21c, third-party clusterware is no longer supported.[1] Network zoning is fundamental to isolate traffic types and enhance security and performance in Oracle RAC. The public network handles client connections and administrative access, requiring gigabit Ethernet or faster links with low latency to support SQL*Net traffic.[17] In contrast, the private interconnect is dedicated to internal cluster communication, such as Cache Fusion for block transfers between instances, and should use a separate high-bandwidth, low-latency network like InfiniBand or 10 Gigabit Ethernet to minimize contention. The private interconnect uses UDP ports (e.g., dynamically assigned in high ranges for Cache Fusion, fixed ports like 1638 for Cluster Synchronization Services). To avoid interference, VLANs are employed for separation, ensuring the private interconnect operates in an isolated segment that prevents public traffic from impacting cluster heartbeat or data synchronization.[18] Security basics in cluster setup focus on node authentication and controlled access to prevent unauthorized participation. The Oracle Advanced Security Option (ASO) enables strong authentication mechanisms, such as Kerberos or PKI-based certificates, to verify node identities during cluster join operations.[19] Firewall configurations must allow specific ports: TCP port 1521 for SQL*Net listener connections from clients, and ensure the private interconnect's UDP ports are open between nodes while blocking external access.[20]Architecture
Key Components
Oracle Clusterware serves as the foundational cluster management software for Oracle Real Application Clusters (RAC), providing a portable infrastructure that binds multiple servers to operate as a single logical system. It manages essential cluster resources, including nodes, virtual IP (VIP) addresses, database services, and listeners, ensuring high availability and automated failover. Key subcomponents include Cluster Ready Services (CRS), which oversees the lifecycle of cluster resources stored in the Oracle Cluster Registry (OCR); Cluster Synchronization Services (CSS), which maintains node membership and synchronizes cluster operations to prevent split-brain scenarios; and Event Management (EVM), which monitors and propagates cluster events for proactive management. These elements collectively enable seamless resource allocation and recovery across the cluster.[3][21] Shared resources form the core data layer in Oracle RAC, allowing multiple database instances to access the same physical storage simultaneously. These include data files, control files, server parameter files (SPFILEs), and redo log files, all residing on cluster-aware shared disks that meet prerequisites such as high-speed access from all nodes. Automatic Storage Management (ASM) instances manage this shared storage, dynamically allocating disk groups and optimizing I/O performance for RAC environments. Oracle Net listeners, configured per node or as shared listeners, facilitate client connections, while the Single Client Access Name (SCAN) enhances scalability by resolving to three IP addresses for connection load balancing and failover across instances.[3][22] Oracle RAC introduces specialized background processes to handle distributed operations, distinguishing it from single-instance databases. Multiple Lock Manager Server (LMS) processes per instance manage buffer cache coordination, including the transfer of data blocks between instances, which is critical for Cache Fusion's reliance on these processes. The Lock Manager Daemon (LMD) process oversees distributed lock management, processing remote enqueue requests, detecting deadlocks, and ensuring resource consistency across the cluster. Additional processes like the Lock Monitor (LMON) support global enqueue monitoring and instance recovery, while the Lock Element (LCK) handles cross-instance calls. These processes operate alongside standard database background processes to maintain data integrity and performance in a multi-node setup.[3] Oracle Grid Infrastructure integrates Oracle Clusterware with ASM, creating a unified platform for automated cluster and storage management in RAC deployments. Installed on each node, it simplifies administration by combining resource management with disk provisioning, supporting up to 128 instances per database and enabling features like dynamic volume management. This integration reduces operational complexity, allowing administrators to treat the cluster as a single entity for provisioning and maintenance tasks.[21][3][23]Cache Fusion Protocol
Cache Fusion is the core protocol in Oracle Real Application Clusters (RAC) that enables efficient synchronization of data blocks across multiple database instances by transferring them directly between buffer caches over the private cluster interconnect, thereby bypassing disk I/O and minimizing latency. This mechanism logically merges the buffer caches of all instances into a single, shared global cache, allowing instances to access data as if it were local without frequent disk contention. By shipping blocks in memory rather than reading from shared storage, Cache Fusion significantly reduces reader/writer conflicts and overall system overhead, enhancing scalability for high-throughput workloads.[3][24][25] The protocol operates in two primary modes to handle read and write requests while maintaining data consistency. For read operations, Cache Fusion transfers consistent read (CR) blocks, which are versions of data blocks constructed to reflect a consistent view at the time of the request, avoiding the need for rollback application on the requesting instance. In write scenarios, it transfers current (dirty) blocks—those modified but not yet written to disk—accompanied by lock mode conversions, such as upgrading from shared (S) to exclusive (X) mode to ensure only one instance can modify the block at a time. These conversions prevent concurrent modifications and preserve block integrity across the cluster.[26][27][25] The Global Cache Service (GCS), a key component integrated with Cache Fusion, manages the modes and locations of data blocks to enforce cache coherency. GCS tracks blocks in modes including null (no holder), shared (multiple readers), exclusive (single modifier), and S/SSX (shared with shared past image for undo reconstruction). It coordinates access privileges by monitoring resource states and facilitating block transfers via Lock Management Services (LMS) processes, which handle inter-instance messaging. Additionally, GCS employs heartbeat mechanisms through these processes to continuously track resource availability and instance liveness, ensuring prompt detection of failures and reallocation of resources.[28][25][29] Performance of Cache Fusion is evaluated using key metrics such asgc cr blocks received and gc cr blocks sent for consistent read transfers, and gc current blocks received and gc current blocks sent for current block transfers, which indicate the volume of inter-instance block traffic. These statistics, available in views like V$SYSSTAT, help identify bottlenecks in global cache activity. Fusion efficiency, often expressed as the ratio of CR blocks to total global cache blocks multiplied by 100, provides a measure of how effectively the protocol avoids disk access, with higher percentages signaling optimal cache utilization.[26][30]
Cache Fusion was introduced in Oracle Database 9i as a groundbreaking feature to enable true shared-cache architecture in RAC, replacing earlier disk-based coordination methods. Subsequent enhancements, particularly from Oracle Database 19c onward, have incorporated support for Remote Direct Memory Access (RDMA) over protocols like RoCE, allowing direct memory-to-memory transfers with reduced CPU involvement and further latency improvements on compatible hardware.[4][31]
Networking and Connectivity
Interconnect Design
The interconnect in Oracle Real Application Clusters (RAC) serves as a dedicated private network that provides a high-bandwidth, low-latency communication pathway between cluster nodes, enabling efficient inter-instance data transfers via Cache Fusion, cluster heartbeats for node membership monitoring, and other essential cluster traffic.[3] This infrastructure is crucial for maintaining data consistency and scalability across multiple database instances, treating the cluster as a unified system.[6] Common implementations utilize 10, 25, or 100 Gigabit Ethernet (GbE) or InfiniBand to meet these performance demands.[3] To ensure high availability, Oracle RAC employs redundancy through dual or multiple interconnect interfaces per node, configured without traditional bonding but leveraging High Availability IP (HAIP) for automatic failover and load balancing.[32] HAIP dynamically assigns virtual IP addresses across available interfaces on different subnets, allowing seamless traffic redirection if a physical link or switch fails, thus preventing single points of failure.[32] The protocol stack primarily uses UDP for its efficiency in handling the bursty, high-volume traffic patterns, with support for Reliable Datagram Sockets (RDS) over InfiniBand as an alternative.[3] Bandwidth requirements for the interconnect start at a minimum of 1 Gb/s, but Oracle recommends 10 Gb/s or higher to support intensive workloads without bottlenecks.[33] Latency must be kept low, ideally under 1 ms round-trip, to optimize Cache Fusion performance and minimize global cache waits.[34] These specifications ensure that the interconnect functions effectively as a shared memory channel equivalent for the distributed database environment.[6] Configuration of the interconnect involves assigning a private IP address range exclusive to cluster communication, separate from public networks, to isolate and prioritize internal traffic.[3] Enabling jumbo frames with a Maximum Transmission Unit (MTU) of 9000 bytes is recommended across all components, including host adapters, drivers, and switches, to reduce overhead and improve throughput for large block transfers.[33] Starting with Oracle Database 12c, integration with RDMA over Converged Ethernet (RoCE) enhances performance by enabling kernel-bypass direct memory access, particularly in environments supporting RDMA protocols like RDSv3.[13]Name Resolution and VIPs
In Oracle Real Application Clusters (RAC), name resolution for client connections occurs over the public network, where clients access database instances without needing to know specific node details. This setup relies on virtual IP addresses (VIPs) and domain name system (DNS) configurations to ensure seamless connectivity and high availability. Node VIPs are floating IP addresses hosted by Oracle Clusterware on the public network interface of each cluster node, allowing clients to connect directly to an instance via these addresses.[35] By using VIPs, Oracle RAC achieves sub-second failover for client connections; if a node fails, the VIP migrates to another available node, preventing the typical TCP timeout delays of up to several minutes that would occur with static IP binding.[36] To simplify client access and enable load balancing across the cluster, Oracle RAC introduced the Single Client Access Name (SCAN) in release 11g. SCAN is a DNS-based alias that resolves to three IP addresses, each associated with a SCAN VIP and listener, distributing incoming connections evenly without requiring clients to specify individual node VIPs.[37] These SCAN VIPs function similarly to node VIPs but can relocate to any node in the cluster, providing a stable, single endpoint for clients regardless of node additions or failures.[38] The SCAN configuration enhances scalability, as it supports up to three addresses for redundancy and load distribution, and integrates with the cluster's public network zoning to maintain consistent access.[39] For automated management of VIPs and hostnames in larger or dynamic environments, Oracle RAC introduced the Grid Naming Service (GNS) in release 11g Release 2 (11.2). GNS operates as a cluster-managed DNS service, using a static GNS VIP address delegated by the external DNS to handle dynamic resolution of cluster resources, including node VIPs and SCAN addresses.[40] This integration reduces administrative overhead by automatically updating DNS records for VIP migrations during node additions, removals, or failures, ensuring clients always resolve to active resources without manual intervention.[41] Client applications connect to Oracle RAC using standard protocols like JDBC or ODBC, configured to target the SCAN name for transparent node affinity and failover. For instance, a JDBC connection string might specify the SCAN hostname and port, allowing the driver to select an optimal instance based on service preferences while receiving Fast Application Notification (FAN) events for real-time updates on cluster changes.[37] FAN, part of the Oracle RAC high availability framework, notifies applications via APIs or callbacks about service relocations or instance failures, enabling proactive reconnection without full session loss.[42] This configuration supports ODBC drivers similarly, promoting workload balancing and rapid recovery in enterprise deployments.[43]Implementation and Operations
Deployment Process
The deployment of Oracle Real Application Clusters (RAC) begins with preparing the operating system on all cluster nodes to meet the necessary prerequisites, such as installing required kernel packages, configuring user accounts like the grid and oracle owners, setting kernel parameters for shared memory and semaphores, and ensuring SSH equivalence for passwordless communication between nodes. These steps ensure compatibility and secure inter-node operations before proceeding to software installation.[44] Next, Oracle Grid Infrastructure is deployed using the Oracle Universal Installer (OUI), which is run on the first node in graphical or silent mode to install Oracle Clusterware and Oracle Automatic Storage Management (ASM) across the cluster.[45] During the installation, OUI prompts for cluster node selection, virtual IP addresses, and storage options; it then copies the software to remote nodes and requires execution of theroot.sh script on each node as the root user to configure system-level components like the Cluster Ready Services (CRS).[46] Following Grid Infrastructure installation, ASM disk groups are created using the ASM Configuration Assistant (ASMCA) or through OUI options to provision shared storage for the database files, voting disks, and OCR.
The Oracle Database software is then installed in RAC mode using OUI in "software only" configuration, selecting the Enterprise Edition and specifying the cluster nodes, with Oracle base and home directories distinct from the Grid home.[47] After software installation, the root.sh script is run on all nodes if not automated during OUI. To create the RAC database, the Database Configuration Assistant (DBCA) is invoked from the Oracle home, choosing the "Oracle Real Application Clusters database" template, specifying instances per node, and configuring parameters like memory and storage allocation to ASM disk groups. DBCA automates the creation of database instances, services, and listeners across nodes.
Cluster resources, including databases and instances, are managed post-deployment using the Server Control (SRVCTL) utility, which allows starting, stopping, and monitoring RAC components from any node.[3] For maintenance, Oracle RAC supports rolling upgrades through a two-stage process: first, upgrading the Grid Infrastructure software node-by-node while maintaining cluster availability, followed by applying patches or database upgrades in batches to minimize downtime.[48] This approach leverages QUORUM-based voting disk management, where a majority of voting disks must remain accessible to sustain cluster operations during the upgrade.[48]
Verification of the deployment ensures all components are operational; the Cluster Verification Utility (CLUVFY) is run with commands like cluvfy stage -post crsinst -n all to check post-installation status of Clusterware and interconnect.[49] Additionally, crsctl check crs confirms the status of Cluster Ready Services, CSS, and EVM daemons on each node, while srvctl status database -d db_name verifies database instance availability across the cluster.[50] These tools provide comprehensive diagnostics to validate the RAC environment before production use.
Management and Diagnostics
Oracle Real Application Clusters (RAC) management involves a suite of command-line utilities and repositories designed to administer cluster resources, instances, and services efficiently. The Server Control Utility (SRVCTL) serves as the primary interface for managing Oracle RAC databases, enabling administrators to start, stop, and relocate database instances, services, and other cluster resources from a centralized command line. For instance, commands likesrvctl start database -d db_name initiate all instances of a specified database across the cluster, while srvctl stop instance -d db_name -i instance_name halts a specific instance without affecting others.[51] Complementing SRVCTL, the Cluster Ready Services Control (CRSCTL) utility provides control over Oracle Clusterware components, allowing operations such as checking cluster status with crsctl check cluster -all, starting or stopping the entire cluster stack via crsctl start crs, or managing high-availability resources.[52] These tools ensure seamless administration of multi-node environments by abstracting complex cluster interactions into simple, scriptable commands.
The Grid Infrastructure Management Repository (GIMR) enhances diagnostics by storing cluster-wide operational data, including performance metrics, alerts, and historical logs from Oracle Clusterware and RAC components, in a dedicated multitenant database pluggable database (PDB) per cluster. Administrators access GIMR data through tools like Oracle Enterprise Manager or SQL queries to diagnose issues such as resource failures or configuration drifts, facilitating proactive maintenance without relying on external storage.
Monitoring in Oracle RAC leverages Automatic Workload Repository (AWR) and Automatic Database Diagnostic Monitor (ADDM) to capture and analyze cluster-specific performance data, with a focus on global cache (gc) wait events that indicate inter-node block transfer delays. AWR snapshots, generated hourly by default, collect statistics on gc cr block receive time (waits for current or consistent read blocks) and gc buffer busy waits (contention for cached blocks), enabling identification of interconnect bottlenecks or load imbalances across instances. ADDM processes these AWR snapshots to produce actionable recommendations, such as optimizing SQL statements contributing to excessive gc waits or adjusting instance parameters for better cache fusion efficiency.[53][54] For real-time visualization, Oracle Enterprise Manager (OEM) offers dashboards that monitor RAC clusters at the node, instance, and service levels, displaying metrics like CPU usage, I/O throughput, and cluster interconnect health through graphical interfaces and alerts. OEM integrates with Clusterware to provide end-to-end views, including proactive notifications for threshold breaches.[55]
Diagnostics in Oracle RAC emphasize automated collection and analysis to expedite issue resolution. The Trace File Analyzer (TFA) automates the gathering of diagnostic logs, traces, and system information from Grid Infrastructure, RAC databases, and supporting OS components, supporting proactive detection of problems like memory leaks or network faults via its collector daemon. Administrators invoke TFA with commands such as tfactl diagcollect to bundle relevant files for support analysis. Oracle Recommended Patch Analyzer (ORAchk) performs comprehensive health checks, scanning configurations, patches, and best practices compliance across the RAC stack to identify vulnerabilities or misconfigurations before they impact availability. It generates reports with severity ratings and remediation steps, often run periodically or on-demand via orachk -runallchecks.[56] Event handling is streamlined through Fast Application Notification (FAN) and Fast Connection Failover (FCF), where FAN notifies applications of cluster events like node failures or service relocations, and FCF enables connection pools to automatically redirect to surviving instances in under a second, minimizing downtime without application code changes.[57]
Common issues in Oracle RAC, such as split-brain scenarios where network partitions lead to multiple nodes attempting concurrent database control, are resolved through Oracle Clusterware's fencing mechanisms, which evict errant nodes via node kill or power-off actions to maintain data integrity and prevent corruption. Interconnect latency, often manifesting as elevated gc waits, is diagnosed using standard OS utilities like ping for round-trip times and traceroute for path analysis on the private cluster network, helping isolate hardware or configuration problems.[58]
