Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Administration and Deployment Guide 10g Release 2 (10.2) Part Number B14197-15 |
|
|
PDF · Mobi · ePub |
This chapter describes how to manage workloads in Oracle Real Application Clusters (Oracle RAC) to provide high availability and scalability for your applications. This chapter contains the following topics:
Introduction to Workload Management and Application High Availability
Oracle Clients that Are Integrated with Fast Application Notification
Services and Distributed Transaction Processing in Oracle RAC
Administering Services with Enterprise Manager, DBCA, PL/SQL, and SRVCTL
Measuring Performance by Service Using the Automatic Workload Repository
Enabling Event Notification for Connection Failures in Oracle Real Application Clusters
Workload management enables you to manage workload distributions to provide optimal performance for users and applications. Workload management comprises the following:
Services: Oracle Database 10g introduces a powerful automatic workload management facility, called services, to enable the enterprise grid vision. Services are entities that you can define in Oracle RAC databases that enable you to group database workloads and route work to the optimal instances that are assigned to offer the service.
Connection Load Balancing: A feature of Oracle Net Services that balances incoming connections across all of the instances that provide the requested database service.
High Availability Framework: An Oracle RAC component that enables the Oracle Database to maintain components in a running state at all times.
Fast Application Notification (FAN): The notification mechanism that Oracle RAC uses to quickly alert applications about configuration and workload service level changes.
Load Balancing Advisory: Provides information to applications about the current service levels that the database and its instances are providing. The load balancing advisory makes recommendations to applications about where to direct application requests to obtain the best service based on the policy that you have defined for that service.
Fast Connection Failover: This is the ability of Oracle Clients to provide rapid failover of connections by subscribing to FAN events.
Runtime Connection Load Balancing: This is the ability of Oracle Clients to provide intelligent allocations of connections in the connection pool based on the current service level provided by the database instances when applications request a connection to complete some work.
When a user or application connects to a database, Oracle recommends that you specify a service in the connect data portion of the connect string. Oracle Database automatically creates one database service when the database is created. For many installations, this may be all you need. To enable more flexibility in the management of the workload using the database, Oracle Database 10g enables you to create multiple services and specify which instances offer the services. Continue reading this chapter to understand the added features that you can use with services if you are interested in greater workload management flexibility.
Note:
The features discussed in this chapter do not work with the default database service. You must create cluster managed services to take advantage of these features. You can only manage services that you create. Any service created by the database server will be managed by the database server.You can deploy Oracle RAC and single-instance Oracle database environments to use workload management features in many different ways. Depending on the number of nodes and your environment's complexity and objectives, your choices for the optimal workload management and high availability configuration depend on several considerations that this chapter describes. The following section describes the various service deployment options.
This section describes the following service deployment topics:
To manage workloads, you can define services that you assign to a particular application or to a subset of an application's operations. You can also group work by type under services. For example, online users can be a service while batch processing can be another and reporting can be yet another service type.
Oracle recommends that all users who share a service have the same service level requirements. You can define specific characteristics for services and each service can be a separate unit of work. There are many options that you can take advantage of when using services. Although you do not have to implement these options, using them helps optimize application performance.
When you define a service, you define which instances normally support that service. These are known as the PREFERRED
instances. You can also define other instances to support a service if the service's preferred instance fails. These are known as AVAILABLE
instances.
When you specify PREFERRED
instances, you are specifying the number of instances on which a service will normally run. The Oracle Clusterware attempts to ensure that the service always runs on the number of nodes for which you have configured the service. Afterwards, due to either instance failure or planned service relocations, a service may be running on an AVAILABLE
instance. When a service moves to an available instance, Oracle does not move the service back to the PREFERRED
instance when the PREFERRED
instance restarts because:
The service is already running on the desired number of instances
Maintaining the service on the current instance provides a higher level of service availability
Not moving the service back to the initial PREFERRED
instance prevents a second outage
You can, however, easily automate fail back by using FAN callouts.
Resource profiles are automatically created when you define a service. A resource profile describes how Oracle Clusterware should manage the service and which instance the service should failover to if the preferred instance stops. Resource profiles also define service dependencies for the instance and the database. Due to these dependencies, if you stop a database, then the instances and services are automatically stopped in the correct order.
Services are integrated with Resource Manager which enables you to restrict the resources that are used by a service within an instance. The Resource Manager enables you to map a consumer group to a service so that users who connect with the service are members of the specified consumer group. Also, the Automatic Workload Repository (AWR) enables you to monitor performance by service.
Oracle Net Services provides connection load balancing to enable you to spread user connections across all of the instances that are supporting a service. For each service, you can define the method that you want the listener to use for load balancing by setting the connection load balancing goal, CLB_GOAL
. You can also specify a single TAF policy for all users of a service by defining the FAILOVER_METHOD
, FAILOVER_TYPE
, and so on. Oracle RAC uses FAN to notify applications about configuration changes and the current service level that is provided by each instance where the service is enabled.
FAN has two methods for publishing events to clients, the Oracle Notification Service (ONS), which is used by Java Database Connectivity (JDBC) clients including the Oracle Application Server 10g, and Oracle Streams, Advanced Queueing which is used by Oracle Call Interface (OCI) and Oracle Data Provider for .NET (ODP.NET) clients. When using Advanced Queueing, you must enable the service to use the queue by setting AQ_HA_NOTIFICATIONS
to true.
With Runtime Connection Load Balancing, applications can use load balancing advisory events to provide better service to users. The Oracle JDBC and ODP.NET clients are automatically integrated to take advantage of load balancing advisory events. The load balancing advisory informs the client about the current service level that an instance is providing for a service. The load balancing advisory also recommends how much of the workload should be sent to that instance. In addition, Oracle Net Services provides connection load balancing to enable you to spread user connections across all of the instances that support a service. To enable the load balancing advisory, set the GOAL parameter on the service.
Distributed transaction processing applications have unique requirements. To make it easier to use Oracle RAC with global transactions, set the distributed transaction processing parameter on the service so that all tightly coupled branches of a distributed transaction processing transaction are run on the same instance.
See Also:
"Services and Distributed Transaction Processing in Oracle RAC" for more information about distributed transaction processing in Oracle RACAs mentioned earlier, DBCA creates a default service for your Oracle RAC database and this service is a special Oracle database service. This default service is always available on all instances in an Oracle RAC environment, unless an instance is in restricted mode. You cannot alter this service or its properties. The database also supports the following two internal services:
SYS$BACKGROUND
is used by the background processes only
SYS$USERS
is the default service for user sessions that are not associated with any application service
Both of these internal services support all of the workload management features. You cannot stop or disable either of these internal services.
Oracle Net Services provides the ability to balance client connections across the instances in an Oracle RAC configuration. There are two types of load balancing that you can implement: client-side and server-side load balancing. Client-side load balancing balances the connection requests across the Listeners. With server-side load balancing, the Listener directs a connection request to the best instance currently providing the service by using the load balancing advisory. In an Oracle RAC database, client connections should use both types of connection load balancing.
FAN, FCF, and the load balancing advisory depend on an accurate connection load balancing configuration that includes setting the connection load balancing goal for the service. You can use a goal of either long or short for connection load balancing. These goals have the following characteristics:
Long: Use the LONG
connection load balancing method for applications that have long-lived connections. This is typical for connection pools and SQL*Forms sessions. Long is the default connection load balancing goal. The following is an example of modifying a service, POSTMAN
, with the PL/SQL DBMS_SERVICE
package and the CLB_GOAL_LONG
package constant to define the connection load balancing goal for long-lived sessions:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'POSTMAN' , clb_goal => DBMS_SERVICE.CLB_GOAL_LONG);
Short: Use the SHORT
connection load balancing method for applications that have short-lived connections. The following example modifies the service known as ORDER
, using the DBMS_SERVICE.CLB_GOAL_SHORT
package constant to set the goal to short:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'ORDER' , CLB_GOAL => DBMS_SERVICE.CLB_GOAL_SHORT);
When you create an Oracle RAC database with the Database Configuration Assistant (DBCA), DBCA configures and enables server-side load balancing by default. The DBCA also creates a sample client-side load balancing connection definition in the tnsnames.ora
file on the server. Any services created through DBCA have the default setting of CLB_GOAL=CLB_GOAL_LONG
.
When Oracle Net Services establishes a connection to an instance, the connection remains open until the client closes the connection, the instance is shutdown, or a failure occurs. If you configure transparent application failover (TAF) for the connection, then Oracle moves the session to a surviving instance when an outage occurs.
TAF can restart a query after failover has completed but for other types of transactions, such as INSERT
, UPDATE
, or DELETE
, the application must rollback the failed transaction and resubmit the transaction. You must re-execute any session customizations, in other words, ALTER SESSION
statements, after failover has occurred. However, with TAF, a connection is not moved during normal processing, even if the workload changes over time.
Services simplify the deployment of TAF. You can define a TAF policy for a service and all connections using this service will automatically have TAF enabled. This does not require any client-side changes. The TAF setting on a service overrides any TAF setting in the client connection definition. To define a TAF policy for a service, use the DBMS_SERVICE
PL/SQL procedure as in the following example:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'gl.us.oracle.com' , aq_ha_notifications => TRUE , failover_method => DBMS_SERVICE.FAILOVER_METHOD_BASIC , failover_type => DBMS_SERVICE.FAILOVER_TYPE_SELECT , failover_retries => 180 , failover_delay => 5 , clb_goal => DBMS_SERVICE.CLB_GOAL_LONG);
Client-side load balancing is defined in your client connection definition by setting the parameter LOAD_BALANCE=ON
(the default is ON
for description lists). When you set this parameter to ON
, Oracle randomly selects an address in the address list, and connects to that node's Listener. This balances client connections across the available Listeners in the cluster. When the Listener receives the connection request, the Listener connects the user to an instance that the Listener knows provides the requested service. To see what services a Listener supports, run the lsnrctl services
command.
See Also:
Oracle Database Net Services Administrator's Guide for detailed information about both types of load balancingThis section provides a detailed description of FAN under the following topics:
Using Fast Application Notification Callouts
See Also:
"Oracle Clients that Are Integrated with Fast Application Notification" for more information about specific client environments that you can use with FANFAN is a notification mechanism that Oracle RAC uses to notify other processes about configuration and service level information such as includes service status changes, such as UP or DOWN events. Applications can respond to FAN events and take immediate action. FAN UP
and DOWN
events can apply to instances, services, and nodes.
For cluster configuration changes, the Oracle RAC high availability framework publishes a FAN event immediately when a state change occurs in the cluster. Instead of waiting for the application to poll the database and detect a problem, applications can receive FAN events and react immediately.
FAN also publishes load balancing advisory events. Applications can take advantage of the load balancing advisory FAN events to direct work requests to the instance in the cluster that is currently providing the best service quality. You can take advantage of FAN events in the following three ways:
Your application can use FAN without programmatic changes if you use an integrated Oracle client. The integrated clients for FAN events include Oracle Database 10g JDBC (Oracle Database 10g Release 2 is required for load balancing), Oracle Database 10g Release 2 ODP.NET, and Oracle Database 10g Release 2 OCI. This includes applications that use TAF.
Applications can use FAN programmatically by using the ONS Application Programming Interface (API) to subscribe to FAN events and to execute event handling actions upon the receipt of an event.
You can implement FAN with server-side callouts on your database tier.
For DOWN
events, the disruption to the application can be minimized because sessions to the failed instance or node can be terminated. Incomplete transactions can be terminated and the application user is immediately notified. Application users who request connections are directed to available instances only. For UP
events, when services and instances are started, new connections can be created so that the application can immediately take advantage of the extra resources. Through server-side callouts, you can also use FAN to:
Log status information
Page DBAs or to open support tickets when resources fail to start
Automatically start dependent external applications that need to be co-located with a service
Change resource plans or to shut down services when the number of available instances decreases, for example, if nodes fail
Automate the fail back of a service to PREFERRED
instances if needed
FAN events are published using ONS and an Oracle Streams Advanced Queuing. The publication mechanisms are automatically configured as part of your Oracle RAC installation.
The Connection Manager (CMAN) and Oracle Net Services Listeners are integrated with FAN events. This enables the Listener and CMAN to immediately de-register services provided by the failed instance and to avoid erroneously sending connection requests to failed instances.
If you use the service goal CLB_GOAL_SHORT
, then the Listener uses the load balancing advisory when the Listener balances the connection loads. When load balancing advisory is enabled, the metrics used for the Listener are finer grained.
Oracle focuses on maintaining service availability. In Oracle RAC, Oracle services are designed to be continuously available with loads shared across one or more instances. The Oracle RAC high availability framework maintains service availability by using Oracle Clusterware and resource profiles.
The Oracle RAC high availability framework monitors the database and its services and sends event notifications using FAN. The Oracle Clusterware recovers and balances services according to business rules.
You can assign services in Oracle RAC to one or more instances. If Oracle RAC detects an outage, then Oracle Clusterware isolates the failed component and recovers the dependent components. For services, if the failed component is an instance, then Oracle Clusterware relocates the service to an available instance in the cluster. FAN events can occur at various levels within the Oracle Server architecture and the response can include notifying external entities by way of ONS, Advanced Queuing, or you can program notification using FAN callouts.
Note:
Oracle does not run Oracle RAC callouts with guaranteed ordering. Callouts are run asynchronously and they are subject to scheduling variabilities.Notification occurs from a surviving node when the failed node is out of service. The location and number of instances in an Oracle RAC environment that provide a service are transparent to applications. Restart and recovery are automatic, including the restarting of the subsystems, such as the Listener and the Automatic Storage Management (ASM) processes, not just the database. You can use FAN callouts to report faults to your fault management system and to initiate repair jobs.
For repairs, upgrades, and changes that require you to isolate one or more instances, Oracle RAC provides interfaces that relocate, disable, and enable services to minimize service disruption to application users. Once you complete the operation, you can return the service to normal operation.
Due to dependencies, if you manually shutdown your database, then all of your services automatically stop. If you then manually restart the database, then you must also restart the database's services. Use FAN callouts to automate starting the services when the database starts.
Table 6-1 describes the FAN event record parameters and the event types, followed by name-value pairs for the event properties. The event type is always the first entry and the timestamp is always the last entry as in the following example:
FAN event type: service_member Properties: version=1.0 service=ERP database=FINPROD instance=FINPROD3 host=node3 status=up
Table 6-1 Event Record Parameters and Descriptions
Parameter | Description |
---|---|
|
Version of the event record. Used to identify release changes. |
|
|
|
The unique database supporting the service; matches the initialization parameter value for |
|
The name of the instance that supports the service; matches the |
|
The name of the node that supports the service or the node that has stopped; matches the node name known to Cluster Synchronization Services (CSS). |
|
The service name; matches the service in |
|
Values are |
|
Failure, Dependency, User, Autostart, Restart. |
|
The number of service members that are currently active; included in all |
|
For node |
|
The local time zone to use when ordering notification events. |
A FAN record matches the database signature of each session as shown in Table 6-2. The signature information is also available using OCI_ATTRIBUTES
. These attributes are available in the OCI Connection Handle. Use this information to take actions on sessions that match the FAN event data.
Table 6-2 FAN Parameters and Matching Database Signatures
FAN Parameter | Matching Oracle Database Signature |
---|---|
|
|
|
|
|
|
|
|
FAN callouts are server-side executables that Oracle RAC executes immediately when high availability events occur. You can use FAN callouts to automate the following activities when events occur in a cluster configuration, such as:
Opening fault tracking tickets
Sending messages to pagers
Sending e-mail
Starting and stopping server-side applications
Maintaining an uptime log by logging each event as it occurs
Relocating low-priority services when high priority services come online
To use FAN callouts, place an executable in the directory CRS_home
/racg/usrco
on every node that runs Oracle Clusterware. If you are using scripts, then set the shell as the first line of the executable. The following is an example file for the CRS_home
/racg/usrco/callout.sh
callout:
#! /bin/ksh FAN_LOGFILE= [your path name]/admin/log/`hostname`_uptime.log echo $* "reported="`date` >> $FAN_LOGFILE &
The following output is from the previous example:
NODE VERSION=1.0 host=sun880-2 incarn=23 status=nodedown reason= timestamp=08-Oct-2004 04:02:14 reported=Fri Oct 8 04:02:14 PDT 2004
See Also:
Table 6-1 for information about the callout and event detailsA FAN record matches the database signature of each session, as shown in Table 6-2. The signature information is also available using OCI_ATTRIBUTES
. These attributes are available in the OCI Connection Handle. Use this information to take actions on sessions that match the FAN event data.
This section describes the load balancing advisory under the following topics:
Load balancing distributes work across all of the available Oracle RAC database instances. Oracle recommends that applications use persistent connections that span the instances that offer a particular service. Connections are created infrequently and exist for a long duration. Work comes into the system with high frequency, borrows these connections, and exists for a relatively short duration. The load balancing advisory provides advice about how to direct incoming work to the instances that provide the optimal quality of service for that work. This minimizes the need to relocate the work later.
By using the THROUGHPUT
or SERVICE_TIME
goals, feedback is built in to the system. Work is routed to provide the best service times globally, and routing responds gracefully to changing system conditions. In a steady state, the system approaches equilibrium with improved throughput across all of the Oracle RAC instances.
Standard architectures that can use the load balancing advisory include connection load balancing, transaction processing monitors, application servers, connection concentrators, hardware and software load balancers, job schedulers, batch schedulers, and message queuing systems. All of these applications can allocate work.
The load balancing advisory is deployed with key Oracle clients, such as a Listener, the JDBC Implicit Connection Cache 10g, and the ODP.NET Connection Pool. The load balancing advisory is also open for third party subscription by way of ONS.
You can configure your environment to use the load balancing advisory by defining service-level goals for each service for which you want to enable load balancing. This enables the load balancing advisory for that service and FAN load balancing events are published. There are two types of service-level goals for runtime:
SERVICE
TIME
: Attempts to direct work requests to instances according to response time. Load balancing advisory data is based on elapsed time for work done in the service plus available bandwidth to the service. An example for the use of SERVICE
TIME
is for workloads such as internet shopping where the rate of demand changes:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'OE' , goal => DBMS_SERVICE.GOAL_SERVICE_TIME - , clb_goal => DBMS_SERVICE.CLB_GOAL_SHORT);
THROUGHPUT
: Attempts to direct work requests according to throughput. The load balancing advisory is based on the rate that work is completed in the service plus available bandwidth to the service. An example for the use of THROUGHPUT
is for workloads such as batch processes, where the next job starts when the last job completes:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'sjob' - , goal => DBMS_SERVICE.GOAL_SERVICE_TIME - , clb_goal => DBMS_SERVICE.CLB_GOAL_LONG);
Setting the goal to NONE
disables load balancing for the service. You can see the goal settings for a service in the data dictionary and in the DBA_SERVICES
, V$SERVICES
, and V$ACTIVE_SERVICES
views.
See Also:
"Administering Services" for more information about administering services and adding goals to servicesThe load balancing advisory FAN events provide metrics for load balancing algorithms. The easiest way to take advantage of these events is to use the Runtime Connection Load Balancing feature of an Oracle integrated client such as JDBC, ODP.NET, or OCI. Client applications can subscribe to these events directly by way of the ONS API. Table 6-3 describes the load balancing advisory FAN event parameters.
Table 6-3 Load Balancing Advisory FAN Events
Parameter | Description |
---|---|
|
Version of the event record. Used to identify release changes. |
|
|
|
The service name; matches the service in |
|
The unique database supporting the service; matches the initialization parameter value for |
|
The name of the instance that supports the service; matches the |
|
The percentage of work requests to send to this database instance. |
|
Indication of the service quality relative to the service goal. Valid values are |
|
The local time zone to use when ordering notification events. |
Use the following example to monitor load balancing advisory events:
SET PAGES 60 COLSEP '|' LINES 132 NUM 8 VERIFY OFF FEEDBACK OFF COLUMN user_data HEADING "AQ Service Metrics" FORMAT A60 WRAP BREAK ON service_name SKIP 1 SELECT TO_CHAR(enq_time, 'HH:MI:SS') Enq_time , user_data FROM sys.sys$service_metrics_tab ORDER BY 1 ;
Oracle has integrated FAN with many of the common client application environments that are used to connect to Oracle RAC databases. Therefore, the easiest way to use FAN is to use an integrated Oracle Client.
You can use the OCI Session Pools, CMAN session pools, and JDBC and ODP.NET connection pools. OCI applications with TAF enabled should use FAN high availability events for fast failover. The overall goal is to enable applications to consistently obtain connections to available instances that provide the best service. Due to the integration with FAN, Oracle integrated clients are more aware of the current status of an Oracle RAC cluster. This prevents client connections from waiting or trying to connect to an instance that is no longer available.
When instances start, Oracle RAC uses FAN to notify the connection pool so that the connection pool can create connections to the recently started instance and take advantage of the additional resources that this instance provides. The use of connection pools and FAN requires that you have properly configured database connection load balancing across all of the instances that provide the service(s) that the connection pool is using. Oracle recommends that you configure both client-side and server-side load balancing with Oracle Net Services, which is the default when you use DBCA to create your database. Oracle connection pools that are integrated with FAN can:
Balance connections across all of the Oracle RAC instances when a service starts; this is preferable to directing the sessions that are defined for the connection pool to the first Oracle RAC instance that supports the service
Remove terminated connections immediately when a service is declared DOWN
at an instance, and immediately when nodes are declared DOWN
Report errors to clients immediately when Oracle detects the NOT RESTARTING
state, instead of making the client wait while the service repeatedly attempts to restart
Balance work requests at run time using load balancing advisory events
The next sections describe how to enable FAN events for the several specific client development environments:
Enabling Java Database Connectivity Clients to Receive FAN Events
Enabling Oracle Call Interface Clients to Receive FAN High Availability Events
Enabling ODP.NET Clients to Receive FAN High Availability Events
Enabling ODP.NET Clients to Receive FAN Load Balancing Advisory Events
Enabling FAN for the Oracle JDBC Implicit Connection Cache enables FAN high availability events in Oracle Database 10g release 1, and the load balancing advisory in Oracle Database 10g release 2. Your application can use the JDBC development environment for either thick or thin JDBC clients to use FAN. You must use the JDBC Implicit Connection Cache to enable the FAN features of Fast Connection Failover and Runtime Connection Load Balancing.
To configure the JDBC client, set the FastConnectionFailoverEnabled
property before making the first getConnection()
request to a data source. When you enable FCF, the failover applies to every connection in the connection cache. If your application explicitly creates a connection cache using the Connection Cache Manager, then you must first set FastConnectionFailoverEnabled
.
See Also:
The Oracle Database JDBC Developer's Guide and Reference for information about configuring the JDBC implicit connection cache and ONSThis procedure explains how to enable FAN events for JDBC. For thick JDBC clients, if you enable FCF, do not enable TAF, either on the client or for the service. The FAN events that you enable can be both high availability events and load balancing advisory events. When using thin or thick JDBC clients, you can use FAN by configuring the JDBC implicit connection cache as described in the following procedure:
On a cache enabled DataSource, set the DataSource property FastConnectionFailoverEnabled
to true
as in the following example to enable FAN for Oracle implicit JDBC connect cache:
OracleDataSource ods = new OracleDataSource() ods.setUser("Scott"); ... ods.setPassword("tiger"); ods.setConnectionCachingEnabled(True); ods.setFastConnectionFailoverEnabled(True); ods.setConnectionCacheName("MyCache"); ods.setConnectionCacheProperties(cp); ods.setURL("jdbc:oracle:thin:@(DESCRIPTION= (LOAD_BALANCE=on) (ADDRESS=(PROTOCOL=TCP)(HOST=VIP1)(PORT=1521)) (ADDRESS=(PROTOCOL=TCP)(HOST=VIP2)(PORT=1521)) (CONNECT_DATA=(service_name=service_name)))");
Note:
Use the following system property to enable FAN without making data source changes:-D oracle.jdbc.FastConnectionFailover=true
.Ensure that you have configured ONS on each node that is running Oracle Clusterware as in the following example using the CRS_home
/opmn/conf/ons.config
file.
Note:
This should have been automatically completed during the Oracle RAC installation.localport=6100 # This is the port ONS is writing to on this node remoteport=6200 # This is the port ONS is listening on this node loglevel=3 useocr=on
The information in the Oracle Cluster Registry (OCR) for ONS daemons is automatically configured by DBCA. If you are upgrading from a previous version of the database, then manually configure the OCR as follows. To add the middle tier nodes or to update the Oracle RAC nodes, use racgons
from the Oracle Clusterware bin
directory:
To add the ONS daemon configuration:
racgons add_config hostname:port [hostname:port] ...
To remove the ONS daemon configuration:
racgons remove_config hostname[:port] [hostname:port]..
Configure remote ONS subscription. Remote ONS subscription offers the following advantages:
Support for an All Java mid-tier software
An ONS daemon is not needed on the client machine; you do not need to manage this process
Simple configuration by way of a DataSource
property
When using remote ONS subscription for FCF, an application uses setONSConfiguration
, using the string remoteONSConfig
, on an Oracle DataSource instance as in the following example:
ods.setONSConfiguration("nodes=racnode1:4200,racnode2:4200");
When you start the application, make sure that the ons.jar
file is located on the application CLASSPATH
. The ons.jar
file is part of the Oracle client installation.
See Also:
Oracle Database JDBC Developer's Guide and Reference for more information about JDBCOracle Call Interface (OCI) clients can register to receive notifications about Oracle RAC high availability events and respond when events occur. This improves the session failover response time in OCI and also removes terminated connections from connection and session pools. This feature works on OCI applications that use TAF, connection pools, or session pools.
First, you must enable a service for high availability events. If your application is using TAF, then enable the TAF settings for the service. Then configure client applications to enable them to connect to an Oracle RAC instance to enable event notification. Clients can register callbacks that are used whenever an event occurs. This reduces the time that it takes to detect a connection failure. During DOWN
event processing, OCI:
Terminates affected connections at the client
Removes connections from the OCI connection pool and the OCI session pool—the session pool maps each session to a physical connection in the connection pool, and there can be multiple sessions for each connection
Fails over the connection if you have configured TAF
If TAF is not configured, then the client only receives an error.
Note:
OCI does not manageUP
events.Perform the following steps to configure Advanced Queueing notifications to an OCI client:
Ensure that the service that you are using has Advanced Queuing notifications enabled by setting the services' values of AQ_HA_NOTIFICATIONS
to TRUE
. For example:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'gl.us.oracle.com' , aq_ha_notifications => TRUE , failover_method => DBMS_SERVICE.FAILOVER_METHOD_BASIC , failover_type => DBMS_SERVICE.FAILOVER_TYPE_SELECT , failover_retries => 180 , failover_delay => 5 , clb_goal => DBMS_SERVICE.CLB_GOAL_LONG);
Enable OCI_EVENTS
at environment creation time on the client as follows:
( OCIEnvCreate(...) )
Client applications must link with the client thread or operating system library.
Clients on which you want to use this functionality that also require notifications, including TAF clients, must:
Register for notifications
Optionally register a client EVENT
callback
An Advanced Queueing ALERT_QUE
is populated automatically. To see the alert information, you can query the views DBA_OUTSTANDING_ALERTS
and DBA_ALERT_HISTORY
. OCI clients obtain a subscription to this queue if they meet the three criteria specified previously in this section.
See Also:
Oracle Call Interface Programmer's Guide for more information about OCIOracle Data Provider for .NET (ODP.NET) connection pools can subscribe to notifications that indicate when nodes, services, and service members are down. After a DOWN
event, Oracle cleans up sessions in the connection pool that go to the instance that stops and ODP.NET proactively disposes connections that are no longer valid. ODP.NET establishes connections to existing Oracle RAC instances if the removal of severed connections brings the total number of connections below the value that is set for the MIN POOL SIZE
parameter.
The procedures for enabling ODP.NET are similar to the procedures for enabling JDBC in that you must set parameters in the connection string to enable FCF. Perform the following steps to enable FAN:
Enable Advanced Queuing notifications by using the DBMS.
SERVICE
package as in the following example:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'gl.us.oracle.com' , aq_ha_notifications => TRUE , clb_goal => DBMS_SERVICE.CLB_GOAL_LONG);
Enable FCF for ODP.NET connection pools by subscribing to FAN high availability events. Do this by setting the ha events
connection string attribute to true
at connection time. Note that this only works if you are using connection pools. In other words, do this if you have set the pooling
attribute to true
, which is the default. The following example shows this in more detail:
// C# using System; using Oracle.DataAccess.Client; class HAEventEnablingSample { static void Main() { OracleConnection con = new OracleConnection(); // Open a connection using ConnectionString attributes // Also, enable "load balancing" con.ConnectionString = "User Id=scott;Password=tiger;Data Source=oracle;" + "Min Pool Size=10;Connection Lifetime=120;Connection Timeout=60;" + "HA Events=true;Incr Pool Size=5;Decr Pool Size=2"; con.Open(); // Create more connections and carry out work against the DB here. // Dispose OracleConnection object con.Dispose(); } }
Use the following procedures to enable ODP.NET to receive FAN load balancing advisory events:
Enable Advanced Queuing notifications by using the DBMS.SERVICE
package as in the following example:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE (service_name => 'gl.us.oracle.com' , aq_ha_notifications => TRUE , failover_method => DBMS_SERVICE.FAILOVER_METHOD_BASIC , failover_type => DBMS_SERVICE.FAILOVER_TYPE_SELECT , failover_retries => 180 , failover_delay => 5 , clb_goal => DBMS_SERVICE.CLB_GOAL_LONG);
Set the GOAL
and CLB_GOAL
for the service and ensure that Oracle Net Service is configured for connection load balancing.
To take advantage of load balancing events with ODP.NET connection pools, set the load balancing string to TRUE
; the default is FALSE
. You can do this at connect time. This only works if you are using connection pools, or when the pooling attribute is set to TRUE
which is the default, as in the following example:
// C# using System; using Oracle.DataAccess.Client; class LoadBalancingEnablingSample { static void Main() { OracleConnection con = new OracleConnection(); // Open a connection using ConnectionString attributes // Also, enable "load balancing" con.ConnectionString = "User Id=scott;Password=tiger;Data Source=oracle;" + "Min Pool Size=10;Connection Lifetime=120;Connection Timeout=60;" + "Load Balancing=true;Incr Pool Size=5;Decr Pool Size=2"; con.Open(); // Create more connections and carry out work against the DB here. // Dispose OracleConnection object con.Dispose(); } }
See Also:
Oracle Data Provider for .NET Developer's Guide for more information about ODP. NET and the Oracle Database PL/SQL Packages and Types Reference for more information about theDBMS_SERVICES
PL/SQL packageNote:
ODP.NET does not support connection re-distribution when a node starts. However, if you have enabled failover on the server-side, then ODP.NET can migrate connections to available instances.All tightly coupled branches of a distributed transaction running on an Oracle RAC database must run on the same instance. Between transactions and between services, transactions can be load balanced across all of the database instances. This is important when designing your application tier for a distributed transaction processing application that runs on an Oracle RAC database. To provide improved application performance and load balancing, it is better to have several groups of smaller application servers with each group directing its transactions to a single service, or set of services, than to have one or two larger application servers.
In addition, connection pools at the application server tier that load balance across multiple connections to an Oracle RAC database must ensure that all tightly-coupled branches of a global distributed transaction run on only one Oracle RAC instance. This is also true in distributed transaction environments using protocols such as X/Open Distributed Transaction Processing (DTP) or the Microsoft Distributed Transaction Coordinator (DTC).
To simplify the deployment of distributed transactions, you can use services to manage DTP environments. By defining the DTP
property of a service, the service is guaranteed to run on one instance at a time in an Oracle RAC database. All global distributed transactions performed through the DTP service are ensured to have their tightly-coupled branches running on a single Oracle RAC instance. This preserves the integrity for distributed transactions and has the following benefits:
The changes are available locally within one Oracle RAC instance when tightly coupled branches need information about changes made by each other
Relocation and failover of services are fully supported for DTP
By using more DTP services than there are Oracle RAC instances, Oracle can balance the load by services across all of the Oracle RAC database instances
To leverage all of the instances in a cluster, create one or more DTP services for each Oracle RAC instance that hosts distributed transactions. Choose one DTP service for one distributed transaction. Choose different DTP services for different distributed transactions to balance the workload among the Oracle RAC database instances. Because all of the branches of a distributed transaction are on one instance, you can leverage all of the instances to balance the load of many DTP transactions through multiple singleton services, thereby maximizing application throughput.
An external transaction manager, such as OraMTS, coordinates DTP/XA transactions. However, an internal Oracle transaction manager coordinates distributed SQL transactions. Both DTP/XA and distributed SQL transactions must use the DTP service in Oracle RAC.
For current and future client implementations, such as those for JDBC, you do not need the invocation to the SYS.DBMS_SYSTEM.DIST_TXN_SYNC
procedure because the OPS_FAILOVER
flag is deprecated. Instead, the server manages the synchronization of in-doubt transactions across the Oracle RAC instances for transaction recovery. However, for backward compatibility you should retain the invocation to the SYS.DBMS_SYSTEM.DIST_TXN_SYNC
procedure. If you add or delete nodes from your cluster database, then you may need to identify and relocate services that you are using for DTP transactions to ensure that you maintain optimum performance levels.
Your transaction manager does not have to use the XA_RECOVER()
call when the transaction manager has enough information about in-flight transactions. The transaction manager uses the XA_RECOVER()
call only if the transaction manager needs to retrieve a list of prepared transactions from the database, in which case heuristically completed transaction information is also returned from the call.
See Also:
Oracle Database Application Developer's Guide - Fundamentals for more information about transaction branch management in Oracle RACFor services that you are going to use for distributed transaction processing, create the service using Enterprise Manager, DBCA, or SRVCTL and define only one instance as the preferred instance. You can have as many AVAILABLE
instances as you want. For example, the following SRVCTL command creates a singleton service for database crm, xa_01.service.us.oracle.com
, whose preferred instance is RAC01
:
srvctl add service -d crm -s xa_01.service.us.oracle.com -r RAC01 -a RAC02, RAC03
Then mark the service for distributed transaction processing by setting the DTP
parameter to TRUE
; the default is FALSE
. Enterprise Manager enables you to set this parameter on the Cluster Managed Database Services: Create Service or Modify Service page. You can also use the DBMS_SERVICE
package to modify the DTP property of the singleton service as follows:
EXECUTE DBMS_SERVICE.MODIFY_SERVICE(service_name =>'xa_01.service.us.oracle.com', DTP=>TRUE);
If, for example, RAC01
which provides service XA_01
fails, then the singleton service that it provided fails over to one of the other instances, such as RAC02
or RAC03
.
If services migrate to other instances after the cold-start of the Oracle RAC database, then you might need to force the relocation of the service to evenly re-balance the load on all of the available hardware. Use data from the GV$ACTIVE_SERVICES
view to determine whether to do this.
See Also:
Oracle Database Application Developer's Guide - Fundamentals for more information about distributed transactions in Oracle RACWhen you create and administer services, you are dividing the work that your database performs into manageable units. The goal of using services is to achieve optimal utilization of your database infrastructure. You can create and deploy services based on business requirements and Oracle can measure the performance for each service. You can define both the application modules within a service as well as the individual actions for a module and monitor thresholds for these actions. This enables you to manage workloads to deliver capacity on demand.
When you create new services for your database, you should define each service's workload management characteristics. The characteristics of a service include:
A unique global name to identify the service.
A Net Service name that a client uses to connect to the service.
The preferred instances where you want the service to be enabled in a normal running environment.
The available instances where the service can be enabled if one of the preferred instances fails.
A service goal that determines whether connections are made to the service based on best service quality (how efficiently a single transaction completes) or best throughput (how efficiently a complete job or long-running query completes), as determined by the load balancing advisory.
An indicator that determines whether the service is used for distributed transactions.
An indicator that determines whether Oracle RAC high availability events are sent to OCI and ODP.NET clients that have registered to receive them through Advanced Queuing.
See Also:
"Enabling Oracle Call Interface Clients to Receive FAN High Availability Events" for more detailsThe characteristics of session failovers such as whether failovers occur, whether sessions can use pre-existing connections on the failover instance, and whether failed over sessions can continue to process interrupted queries. The service definition can also define the number of times that a failed session attempts to reconnect to the service and how long it should wait between reconnection attempts. The service definition can also include a connection load balancing goal that informs the Listener how to balance connection requests across the instances that provide the service.
The method for load balancing connections for each service. This method is used by the Listener when Oracle creates connections. Connections are classified as LONG
(such as connection pools and SQL*FORMS) which tells the Listener to use session based, or SHORT
which tells the Listener to use CPU based. If load balancing advisory is enabled, its information will be use to balance connections otherwise CPU utilization is used.
In addition to creating services, you may need to:
Delete a service. You can delete services that you created. However, you cannot delete or modify the properties of the default database service that Oracle created.
Check the status of a service. A service can be assigned different roles among the available instances. In a complex database with many services, you may not remember the details of every service. Therefore, you may need to check the status on an instance or service basis. For example, you may need to know the status of a service for a particular instance before you make modifications to that instance or to the Oracle home from which it runs.
Start or stop a service for a database or an instance. A service must be started before it can be used for client connections to that instance. If you shutdown your database, for example, by running the Server Control (SRVCTL) command srvctl stop database -d
database_name
where database_name
is the name of the database that you want to stop, then Oracle stops all services to that database. You must manually re-start the services when you start the database.
Map a Service to a Consumer Group: Oracle can automatically map services to Resource Manager Consumer groups to limit the amount of resources that services can use in an instance. You must create the consumer group and then map the service to the consumer group.
See Also:
Oracle Database PL/SQL Packages and Types Reference for information about theDBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING_PRI
procedureEnable or disable a service for a database or an instance. By default, Oracle Clusterware attempts to restart a service automatically after failures. You can prevent this behavior by disabling a service. Disabling a service is useful when you need to perform database or instance maintenance, for example, if you are performing an upgrade and you want to prevent connection requests.
Relocate a service to a different instance. You can move a service from one instance to another instance to re-balance workloads, for example, after adding or deleting cluster nodes.
Notes:
When you use services, do not set a value for the SERVICE_NAMES
parameter; Oracle controls the setting for this parameter for the services that you create and for the default database service. The service features that this chapter describes are not directly related to the features that Oracle provides when you set SERVICE_NAMES
. In addition, setting a value for this parameter may override some of the benefits of using services.
You must still use INSTANCE_GROUPS
and PARALLEL_INSTANCE_GROUPS
to restrict parallel execution processing to a subset of instances in an Oracle RAC database.
Using service names to access a queue provides location transparency for the queue within an Oracle RAC database.
You can create and administer services with Enterprise Manager, DBCA, and the DBMS_SERVICE
PL/SQL package, and you can perform most service administration tasks with the SRVCTL utility. The following sections describe how to perform service-related tasks using these tools:
The Cluster Managed Database Services page is the master page for beginning all tasks related to services. To access this page, go to the Cluster Database Maintenance page, then click Cluster Managed Database Services in the Services section. You can use this page and drilldowns from this page to:
View a list of services for the clusterView the instances on which each service is currently runningView the status for each serviceCreate or edit a serviceStart or stop a serviceEnable or disable a servicePerform instance-level tasks for a serviceDelete a service
See Also:
Oracle Enterprise Manager Concepts for more information about administering services with Enterprise ManagerYou can perform service-related tasks as described for the following Enterprise Manager pages:
The Cluster Managed Database Services page enables you to:
View a list of services for the cluster, the instances on which each service is currently running, and the status for each service
Start or stop a service, or enable or disable a service
Access the Create Service and Edit Service pages
Access the Services Detail page to perform instance-level tasks for a service
Test the connection for a service
The Cluster Managed Database Services Detail page enables you to:
View the status of a service on all of its preferred and available instances; the status can be Running
, Stopped
, or Disabled
Stop or start a service for an instance of a cluster database
Disable or enable a service for an instance of a cluster database
Relocate a service to manually re-balance the services load
The Create Services page enables you to:
Create the service with name, high availability and performance attributes
Select the desired service policy for each instance configured for the cluster database
Select the desired service properties, such as the TAF policy. (This configures client-side TAF. To set the service TAF policy, use the DBMS_SERVICE
package). You can also set the notification properties, load balancing goals, alert levels, and resource management properties
To access the Cluster Managed Database Services page and detail pages for service instances:
From the Cluster Database Home page, click the Maintenance tab.
From the Cluster Database Maintenance page, under the Services heading in the High Availability options list, click Cluster Managed Database Services. The Cluster and Database Login page appears.
Enter credentials for the database and for the cluster that hosts the cluster database and click Continue. The Cluster Managed Database Services page appears and displays services that are available on the cluster database instances. For information about performing tasks on this page, refer to the online help for this page.
Note:
You must have SYSDBA credentials to access a cluster database. Cluster Managed Database Services does not permit you to connect as anything other than SYSDBA.To manage a service at the instance level, either click a service name or select a service name, then select Manage from the Actions drop-down list and click Go. The Cluster Managed Database Services Detail page for the service appears. For information about performing tasks on this page, refer to the online help for this page.
To access the Relocate page:
Perform steps 1 - 3 from the previous procedure set.
From the Cluster Managed Database Services page, either click a Service name or select a service name, then select Manage from the Actions drop-down list and click Go. The Cluster Managed Database Services Detail page for the service appears.
Click Relocate. The Relocate Service from Instance page appears, where you can perform manual load balancing of services. For information about performing tasks on this page, refer to the online help for this page.
You can administer services in Oracle RAC with DBCA as described under the following topics:
Using the Database Configuration Assistant to Add and Modify Services
Using the Database Configuration Assistant to Administer Services
You can use DBCA to create services and to perform administrative tasks on them. If you use the Database Upgrade Assistant (DBUA) to upgrade a Primary/Secondary configuration from an earlier release of the Oracle database, then the Database Upgrade Assistant (DBUA) creates one service and assigns it to one instance as a preferred instance and to the other instance as an available instance.
See Also:
Oracle Database PL/SQL Packages and Types Reference for more information about theDBMS_SERVICE
PL/SQL and DBMS_MONITOR
packages and for more information about setting thresholdsThe DBCA Services Management feature enables you to manage service assignments and service preferences for instances and to configure TAF policies. You can perform these procedures while your Oracle RAC database is running. Even if your instances or the Oracle RAC database is not running, you can still use DBCA to configure services, but the services will not start automatically. To add, modify, or delete services using the DBCA Services Management feature:
On the DBCA Welcome page, select the Oracle Real Application Clusters option and click Next.
On the DBCA Operations page, select Services Management and click Next.
On the DBCA List of Databases page, select the cluster database for which you want to configure services and click Next. If the database that you selected already has services assigned to it, then DBCA displays these on this page.
Click Add, Modify, or Delete.
To add a service, enter the name of the service. Note that service names with the prefix SYS$
are reserved for use by Oracle internal processes. To modify a service, edit the service and configure the service's instance preferences and TAF policies. Assign the service to instances for preferred (normal) and available (recovery) processing. The DBCA records your changes when you select another service or proceed to another page. To delete a service, select the service and click Delete.
To remove a service, select the service and click Remove.
Click Finish and DBCA displays the Summary page. Click OK and DBCA displays a progress dialog while it configures your services.
When you click Finish, DBCA configures the Oracle Clusterware resources for the services that you added, modified, or removed. The DBCA also configures the net service entries for these services and starts them. When you use DBCA to remove services, DBCA stops the service, removes the Oracle Clusterware resource for the service, and removes the net service entries.
You can manage services with the PL/SQL DBMS_SERVICE
package procedures. These procedures are described in the following sections:
The CREATE_SERVICE
procedure adds a new service to the Oracle RAC database. In the CREATE_SERVICE
syntax service_name
is the unique, global service name, network_name
is the TNS name for connections to the service, goal
sets the workload management goal directive to service quality or throughput, dtp
is the distributed transaction flag, aq_ha_notification
is the flag to send Oracle RAC high availability events to registered OCI clients, failover_method
is the TAF failover method for the service, failover_type
is the TAF failover method for the service, failover_retries
is the TAF connection retry count, failover_delay
is the wait time between TAF connection retries, and clb_goal
sets the connection load balancing goal. When using services with Oracle RAC, add the high availability properties, such as the PREFERRED
and AVAILABLE
placement, using Enterprise Manager, DBCA or SRVCTL.
Note:
Oracle recommends that you use Enterprise Manager or DBCA to create services for Oracle RAC environments.See Also:
Oracle Database PL/SQL Packages and Types Reference for more information about theCREATE_SERVICE
procedure syntax and for information about other procedures mentioned in this sectionThe MODIFY_SERVICE
procedure changes one or more service characteristics. In the MODIFY_SERVICE syntax
the parameters are the same as those for the CREATE_SERVICE
procedure.
The DELETE_SERVICE
procedure drops an existing service. In the DELETE_SERVICE
syntax service_name
is the name of the service to be dropped.
The START_SERVICE
procedure starts a service on the connected instance, the named instance, or all instances. In the START_SERVICE
syntax SERVICE_NAME
is the name of the service to be started and INSTANCE_NAME
can be NULL
(to start the service on the connected instance), the name of an instance (to start the service on that instance), or the package constant DBMS_SERVICE.ALL_INSTANCES
to start the service on all instances where it is defined.
The STOP_SERVICE
procedure stops a service on the connected instance, the named instance, or all instances. In the STOP_SERVICE
syntax the parameters are the same as those for the START_SERVICE
procedure.
The DISCONNECT_SESSION
procedure terminates all sessions on the connected instance, or the instance where the PL/SQL procedure is run. In the DISCONNECT_SESSION
syntax service_name
is the name of the service for which connections are to be dropped.
When you create a service with SRVCTL, you must start it with a separate SRVCTL command. However, you may later need to manually stop or restart the service. You may also need to disable the service to prevent automatic restarts, to manually relocate the service, or obtain status information about the service. The following sections explain how to use SRVCTL to perform the following administrative tasks:
See Also:
Appendix E, " Server Control Utility Reference" for more information about SRVCTL and the other SRVCTL commands that you can use to manage servicesTo create a service with SRVCTL, enter a command from the command line using the following syntax:
srvctl add service -d database_unique_name -s service_name -r preferred_list[-a available_list] [-P TAF_policy]
Enter the following SRVCTL syntax from the command line:
srvctl start service -d database_unique_name [-s service_name_list] [-i inst_name] [-o start_options] [-c connect_str | -q]
srvctl stop service -d database_unique_name -s service_name_list [-i inst_name] [-o start_options] [-c connect_str | -q]
Use the following SRVCTL syntax from the command line to enable and disable services:
srvctl enable service -d database_unique_name -s service_name_list [-i inst_name]
srvctl disable service -d database_unique_name -s service_name_list [-i inst_name]
Run the srvctl relocate service
command from the command line to relocate a service. For example, the following command relocates the crm
service from instance apps1 to instance apps3:
srvctl relocate service -d apps -s crm -i apps1 -t apps3
Run the srvctl relocate service
command from the command line to obtain the status of a service. For example, the following command returns the status of the crm
service that is running on the crm database:
srvctl status service -d apps -s crm
Run the srvctl
relocate
service
command from the command line to obtain the high availability configuration of a service. For example, the following command returns the configuration of the crm
service that is running on the crm
database:
srvctl config service -d apps -s crm -a
See Also:
Appendix E, " Server Control Utility Reference" for information about other administrative tasks that you can perform with SRVCTLServices add a new dimension for performance tuning. With services, workloads are visible and measurable and resource consumption and wait times are attributable by application. Tuning by using 'service and SQL' replaces tuning by 'session and SQL' in the majority of systems where all sessions are anonymous and shared.
The AWR maintains performance statistics that include information about response time, throughput, resource consumption, and wait events for all services and work that a database performs. Oracle also maintains metrics, statistics, wait events, wait classes, and SQL-level traces for services. You can optionally augment these statistics by defining modules within your application to monitor certain statistics. You can also define the actions within those modules that business critical transactions should execute in response to particular statistical values. Enable module and action monitoring using the DBMS_MONTIOR
PL/SQL package as follows:
EXECUTE DBMS_MONITOR.SERV_MOD_ACT_STAT_ENABLE(SERVICE_NAME => 'ERP', MODULE_NAME=> 'PAYROLL', ACTION_NAME => 'EXCEPTIONS PAY'); EXECUTE DBMS_MONITOR.SERV_MOD_ACT_STAT_ENABLE(SERVICE_NAME => 'ERP', MODULE_NAME=>'PAYROLL', ACTION_NAME => NULL);
Use the DBA_ENABLED_AGGREGATIONS
view to verify that you have enabled monitoring.
Statistics aggregation and tracing by service are global in scope for Oracle RAC databases. In addition, they are persistent across instance restarts and service relocations for both Oracle RAC and single-instance Oracle databases.
The service, module, and action names are visible in V$SESSION
, V$ACTIVE_SESSION_HISTORY
, and V$SQL views. The call times and performance statistics are visible in V$SERVICE_STATS
, V$SERVICE_EVENTS
, V$SERVICE_WAIT_CLASSES
, V$SERVICEMETRIC
, and V$SERVICEMETRIC_HISTORY
. When you enable statistics collection for an important transaction, you can see the call speed for each service, module, and action name at each database instance using the V$SERV_MOD_ACT_STATS
view.
The following sample SQL*Plus script provides service quality statistics every five seconds. You can use these service quality statistics to monitor the quality of a service, to direct work, and to balance services across Oracle RAC instances:
SET PAGESIZE 60 COLSEP '|' NUMWIDTH 8 LINESIZE 132 VERIFY OFF FEEDBACK OFF COLUMN service_name FORMAT A20 TRUNCATED HEADING 'Service' COLUMN begin_time HEADING 'Begin Time' FORMAT A10 COLUMN end_time HEADING 'End Time' FORMAT A10 COLUMN instance_name HEADING 'Instance' FORMAT A10 COLUMN service_time HEADING 'Service Time|mSec/Call' FORMAT 999999999 COLUMN throughput HEADING 'Calls/sec'FORMAT 99.99 BREAK ON service_name SKIP 1 SELECT service_name , TO_CHAR(begin_time, 'HH:MI:SS') begin_time , TO_CHAR(end_time, 'HH:MI:SS') end_time , instance_name , elapsedpercall service_time , callspersec throughput FROM gv$instance i , gv$active_services s , gv$servicemetric m WHERE s.inst_id = m.inst_id AND s.name_hash = m.service_name_hash AND i.inst_id = m.inst_id AND m.group_id = 10 ORDER BY service_name , i.inst_id , begin_time ;
Service level thresholds enable you to compare achieved service levels against accepted minimum required levels. This provides accountability with respect to the delivery or the failure to deliver an agreed service level. The end goal is a predictable system that achieves service levels. There is no requirement to perform as fast as possible with minimum resource consumption; the requirement is to meet the quality of service.
You can explicitly specify two performance thresholds for each service: the response time for calls, or SERVICE_ELAPSED_TIME
, and the CPU time for calls, or SERVICE_CPU_TIME
. The response time goal indicates that the elapsed time should not exceed a certain value, and the response time represents wall clock time. Response time is a fundamental measure that reflects all delays and faults that might be blocking the call from running on behalf of the user. Response time can also indicate differences in node power across the nodes of an Oracle RAC database. The service time and CPU time are calculated as the moving average of the elapsed, server-side call time. The AWR monitors the service time and CPU time and publishes AWR alerts when the performance exceeds the thresholds. You can then respond to these alerts by changing the priority of a job, stopping overloaded processes, or by relocating, expanding, shrinking, starting or stopping a service. This permits you to maintain service availability despite changes in demand.
To check the thresholds for the payroll service, use the AWR report. You should record output from the report over several successive intervals during which time the system is running optimally. For example, assume that for an Email server, the AWR report runs each Monday during the peak usage times of 10am to 2pm. The AWR report would contain the response time, or DB time, and the CPU consumption time, or CPU time, for calls for each service. The AWR report would also provide a breakdown of the work done and the wait times that are contributing to the response times.
Using DBMS_MONITOR
, set a warning threshold for the payroll service at 0.5 seconds and a critical threshold for the payroll service at 0.75 seconds. In Oracle Database 10g, you must set these thresholds at all instances within an Oracle RAC database. You can schedule actions using Enterprise Manager jobs for alerts, or you can schedule actions to occur programmatically when the alert is received. In this example, thresholds are added for the servall
service and set as follows:
EXECUTE DBMS_SERVER_ALERT.SET_THRESHOLD( METRICS_ID => DBMS_SERVER_ALERT.ELAPSED_TIME_PER_CALL , warning_operator => DBMS_SERVER_ALERT.OPERATOR_GE , warning_value => '500000' , critical_operator => DBMS_SERVER_ALERT.OPERATOR_GE , critical_value => '750000' , observation_period => 30 , consecutive_occurrences => 5 , instance_name => NULL , object_type => DBMS_SERVER_ALERT.OBJECT_TYPE_SERVICE , object_name => 'servall');
Verify the threshold configuration using the following SELECT statement:
SELECT METRICS_NAME, INSTANCE_NAME, WARNING_VALUE, CRITICAL_VALUE, OBSERVATION_PERIOD FROM dba_thresholds ;
You can enable performance data tracing for important modules and actions within each service. The performance statistics are available in the V$SERV_MOD_ACT_STATS
view. As an example, set the following:
Under the ERP
service, enable monitoring for the exceptions pay action in the module, payroll.
Under the ERP
service, enable monitoring for the all actions in the module, payroll.
Under the HOT_BATCH
service, enable monitoring for the all actions in the module, posting.
EXECUTE DBMS_MONITOR.SERV_MOD_ACT_STAT_ENABLE(service_name => 'erp', module_name=> 'payroll', action_name => 'exceptions pay'); EXECUTE DBMS_MONITOR.SERV_MOD_ACT_STAT_ENABLE(service_name => 'erp', module_name => 'payroll', action_name => NULL); EXECUTE DBMS_MONITOR.SERV_MOD_ACT_STAT_ENABLE(service_name => 'hot_batch', module _name =>'posting', action_name => NULL);
Verify the enabled service, module, action configuration with the following SELECT
statement:
COLUMN AGGREGATION_TYPE FORMAT A21 TRUNCATED HEADING 'AGGREGATION' COLUMN PRIMARY_ID FORMAT A20 TRUNCATED HEADING 'SERVICE' COLUMN QUALIFIER_ID1 FORMAT A20 TRUNCATED HEADING 'MODULE' COLUMN QUALIFIER_ID2 FORMAT A20 TRUNCATED HEADING 'ACTION' SELECT * FROM DBA_ENABLED_AGGREGATIONS ;
The output might appear as follows:
AGGREGATION SERVICE MODULE ACTION ------------ -------------------- ---------- ------------- SERVICE_MODULE_ACTION ERP PAYROLL EXCEPTIONS PAY SERVICE_MODULE_ACTION ERP PAYROLL SERVICE_MODULE_ACTION HOT_BATCH POSTING
Event notification is enabled if the SQL_ORCLATTR_FAILOVER_CALLBACK
and SQL_ORCLATTR_FAILOVER_HANDLE
attributes of the SQLSetConnectAttr
function are set when a connection failure occurs in an Oracle RAC Database environment. Both attributes are set using the SQLSetConnectAttr
function. The symbols for the new attributes are defined in the sqora.h
file. The SQL_ORCLATTR_FAILOVER_CALLBACK
attribute is used to specify the address of a routine to call when a failure event takes place.
The SQL_ORCLATTR_FAILOVER_HANDLE
attribute is used to specify a context handle which will be passed as one of the parameters in the callback routine. This attribute is necessary in order for the ODBC application to determine which connection the failure event is taking place on.
The function prototype for the callback routine is as follows:
void failover_callback(void *handle, SQLINTEGER fo_code)
The handle
parameter is the value that was set by the SQL_ORCLATTR_FAILOVER_HANDLE
attribute. Null is returned if the attribute has not been set.
The fo_code
parameter identifies the failure event that is taking place. The failure events map directly to the events defined in the OCI programming interface. The list of possible events is as follows:
ODBC_FO_BEGIN
ODBC_FO_ERROR
ODBC_FO_ABORT
ODBC_FO_REAUTH
ODBC_FO_END
The following is a sample program that demonstrates how to use this feature:
/* NAME ODBCCallbackTestDESCRIPTION Program to demonstrate the connection failover callback feature.PUBLIC FUNCTION(S) mainPRIVATE FUNCTION(S)NOTESCommand Line: ODBCCallbackTest filename [odbc-driver]*/ #include <malloc.h> #include <stdio.h> #include <string.h> #include <sql.h> #include <sqlext.h> #include <sqora.h> #define TRUE 1 #define FALSE 0 /* * Funtion Prototypes */ void display_errors(SQLSMALLINT HandleType, SQLHANDLE Handle); void failover_callback(void *Handle, SQLINTEGER fo_code); /* * Macros */ #define ODBC_STS_CHECK(sts) \ if (sts != SQL_SUCCESS) \ { \ display_errors(SQL_HANDLE_ENV, hEnv); \ display_errors(SQL_HANDLE_DBC, hDbc); \ display_errors(SQL_HANDLE_STMT, hStmt); \ return FALSE; \ } /* * ODBC Handles */ SQLHENV *hEnv = NULL; // ODBC Environment Handle SQLHANDLE *hDbc = NULL; // ODBC Connection Handle SQLHANDLE *hStmt = NULL; // ODBC Statement Handle /* * MAIN Routine */ main(int argc, char **argv) { SQLRETURN rc; /* * Connection Information */ SQLTCHAR *dsn = "odbctest"; SQLTCHAR *uid = "scott"; SQLTCHAR *pwd = "tiger"; SQLTCHAR *szSelect = "select * from emp"; /* * Allocate handles */ rc = SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, (SQLHANDLE *)&hEnv); ODBC_STS_CHECK(rc) rc = SQLSetEnvAttr(hEnv, SQL_ATTR_ODBC_VERSION, (SQLPOINTER)SQL_OV_ODBC3, 0); ODBC_STS_CHECK(rc); rc = SQLAllocHandle(SQL_HANDLE_DBC, hEnv, (SQLHANDLE *)&hDbc); ODBC_STS_CHECK(rc); /* * Connect to the database */ rc = SQLConnect(hDbc, dsn, (SQLSMALLINT)strlen(dsn), uid, (SQLSMALLINT)strlen(uid), pwd, (SQLSMALLINT)strlen(pwd)); ODBC_STS_CHECK(rc); /* * Set the connection failover attributes */ rc = SQLSetConnectAttr(hDbc, SQL_ORCLATTR_FAILOVER_CALLBACK, &failover_callback,0); ODBC_STS_CHECK(rc); rc = SQLSetConnectAttr(hDbc, SQL_ORCLATTR_FAILOVER_HANDLE, hDbc, 0); ODBC_STS_CHECK(rc); /* * Allocate the statement handle */ rc = SQLAllocHandle(SQL_HANDLE_STMT, hDbc, (SQLHANDLE *)&hStmt); ODBC_STS_CHECK(rc); /* * Wait for connection failovers */ while (TRUE) { sleep(5000); rc = SQLExecDirect(hStmt,szSelect, strlen(szSelect)); ODBC_STS_CHECK(rc); rc = SQLFreeStmt(hStmt, SQL_CLOSE); ODBC_STS_CHECK(rc); } /* * Free up the handles and close the connection */ rc = SQLFreeHandle(SQL_HANDLE_STMT, hStmt); ODBC_STS_CHECK(rc); rc = SQLDisconnect(hDbc); ODBC_STS_CHECK(rc); rc = SQLFreeHandle(SQL_HANDLE_DBC, hDbc); ODBC_STS_CHECK(rc); rc = SQLFreeHandle(SQL_HANDLE_ENV, hEnv); ODBC_STS_CHECK(rc); return TRUE; } /* * Failover Callback Routine */ void failover_callback(void *Handle, SQLINTEGER fo_code) { switch (fo_code) { case ODBC_FO_BEGIN: printf("ODBC_FO_BEGIN recevied"); break; case ODBC_FO_ERROR: printf("ODBC_FO_ERROR recevied"); break; case ODBC_FO_ABORT: printf("ODBC_FO_ABORT recevied"); break; case ODBC_FO_REAUTH: printf("ODBC_FO_REAUTH recevied"); break; case ODBC_FO_END: printf("ODBC_FO_END recevied"); break; default: printf("Invalid or unknown ODBC failover code recevied"); break; }; return; } /* * Retrieve the errors associated with the handle passed * and display them. */ void display_errors(SQLSMALLINT HandleType, SQLHANDLE Handle) { SQLTCHAR MessageText[256]; SQLTCHAR SqlState[5+1]; SQLSMALLINT i=1; SQLINTEGER NativeError; SQLSMALLINT TextLength; SQLRETURN sts = SQL_SUCCESS; if (Handle == NULL) return; /* * Make sure all SQLState text is null terminated */ SqlState[5] = '\0'; /* * Fetch and display all diagnostic records that exist for this handle */ while (sts == SQL_SUCCESS) { NativeError = 0; TextLength = 0; sts = SQLGetDiagRec(HandleType, Handle, i, SqlState, &NativeError,(SQLTCHAR *)&MessageText, sizeof(MessageText),&TextLength); if (sts == SQL_SUCCESS) { printf("[%s]%s\n",NativeError, &MessageText); if (NativeError != 0) { printf("Native Error Code: %d", NativeError); } i++; } } return; }