3 Storage Pre-Installation Tasks

Disabling Write Caching

You must disable write caching on all disks that will be used to share data between nodes in your cluster. To disable write caching, perform these steps:

1. Click Start, then click Settings, then Control Panel, then Administrative Tools, then Computer Management, then Device Manager, and then Disk drives

2. Expand the Disk drives and double-click the first drive listed

3. Under the Disk Properties tab for the selected drive, uncheck the option that enables the write cache

4. Double-click each of the other drives listed in the Disk drives hive and disable the write cache as described in the previous step

Enabling Automounting for Windows 2003

If you are using Windows 2003, then you must enable disk automounting, depending on the Oracle products you are installing and on other conditions.

You must enable automounting when using:

• Raw partitions for Oracle Real Application Clusters (Oracle RAC)

• Cluster file system for Oracle RAC

• Oracle Clusterware

• Raw partitions for a single-node database installation

• Logical drives for Automatic Storage Management (ASM)

To enable automounting:

1. Enter the following commands at a command prompt:

   c:\> diskpart
   DISKPART> automount enable
   Automatic mounting of new volumes enabled.
   

2. Type exit to end the diskpart session

3. Repeat steps 1 and 2 for each node in the cluster.

4. When you have prepared all the cluster nodes in your Windows 2003 system as described in the previous steps, restart all of the nodes.

Overview of Storage Options

Use the following overview to help you select your storage option:

https://metalink.oracle.com

Checking for Available Shared Storage with CVU

To check for all shared file systems available across all nodes on the cluster, use the following CVU command:

cluvfy comp ssa -n node_list

Remember to use the full path name and the runcluvfy.bat command on the installation media and include the list of nodes in your cluster, separated by commas, for the node_list. The following example is for a system with two nodes, node1 and node2, and the installation media on drive F:

F:\> clusterware\cluvfy\runcluvfy.bat comp ssa -n node1,node2

If you want to check the shared accessibility of a specific shared storage type to specific nodes in your cluster, then use the following command syntax:

cluvfy comp ssa -n node_list -s storageID_list

In the preceding syntax, the variable node_list is the list of nodes you want to check, separated by commas, and the variable storageID_list is the list of storage device IDs for the storage devices managed by the file system type that you want to check.

Configuring Storage for Oracle Clusterware Files on a Shared File System

If you plan to use OCFS for your Oracle home and datafiles, then the following partitions must exist before you run OUI to install Oracle Clusterware:

• 3 GB, or larger, for the Oracle home

• 3 GB, or larger, for the datafiles

The OCR and voting disk, required by Oracle Clusterware, are also stored in the OCFS datafile directory (datafile_disk\cdata\clustername) where datafile_disk is the OCFS partition and clustername is the name of your cluster.

Oracle Universal Installer (OUI) does not suggest a default location for the Oracle Cluster Registry (OCR) or the Oracle Clusterware voting disk. If you choose to create these files on a file system, then perform the steps described in this section to set up the shared disk raw partitions for OCFS. Windows refers to raw partitions as logical drives. If you need more information about creating partitions, then refer to the Windows online help from within the disk administration tools.

1. Run Windows Disk Management from one node to create an extended partition. Use a basic disk: dynamic disks are not supported.

2. Create at least two partitions: one for the Oracle home and one for the Oracle database files.

   You do not need to create a partition for the voting disk if you plan to use OCFS. OCFS stores the voting device in a file.

   The number of partitions used for OCFS affects performance. Therefore, you should create the minimum number of partitions needed for the OCFS option you choose.

To create the required partitions, perform the following steps:

1. From one of the existing nodes of the cluster, run the Windows disk administration tool as follows:

   • Click Start, then select Settings, Control Panel, Administrative Tools, and then Computer Management

   • Expand the Storage folder to Disk Management. Use a basic disk with a Master Boot Record (MBR) partition style as an extended partition for creating partitions.

2. Right click inside an unallocated part of an extended partition and choose Create Logical Drive. A wizard presents pages for configuring the logical drive. Select the select logical drive radio button and click Next.

3. Enter the size that you want for the partition and click Next.

4. Choose the option "Do not assign a drive letter or path", click Next, and then choose the option "Do not format this partition". Click Finish on the last page of the wizard.

5. Repeat steps 2 through 4 for the second and any additional partitions. An optimal configuration is one partition for the Oracle home and one partition for Oracle database files.

6. If you are preparing drives on a Windows 2003 system, then you should restart all nodes in the cluster after you have created the logical drives.

7. Check all nodes in the cluster to ensure that the partitions are visible on all the nodes and to ensure that none of the Oracle partitions have drive letters assigned. If any partitions have drive letters assigned, then remove them by performing these steps:

   • Right-click the partition in the Windows disk administration tool

   • Select "Change Drive Letters and Paths..." from the menu

   • Click Remove in the "Change Drive Letter and Paths" window

Configuring Storage for Oracle Clusterware Files on Raw Devices

To use raw devices, create two partitions: One 256MB partition for the voting disk and one 256 MB partition for the OCR. If you are not using OCFS for your datafiles, then you must also create raw partitions for your database files as described in "Configuring Raw Logical Volumes or Raw Partitions".

General Steps for Configuring Automatic Storage Management

You will follow these general steps to configure Automatic Storage Management:

1. Identify your site's storage requirements.

2. Optionally, use an existing Automatic Storage Management disk group.

3. If you are creating a new Automatic Storage Management disk group, then create partitions for DAS or SAN disks.

4. Use one of the following methods to complete the Automatic Storage Management configuration:

   • If you plan to install Oracle Database using interactive mode, then Oracle Universal Installer prompts you for the Automatic Storage Management disk configuration information during the installation.

   • If you plan to install Oracle Database using noninteractive mode, then you must configure the disks manually before performing the installation.

Step 1: Identifying Storage Requirements for Automatic Storage Management

To identify the storage requirements for using Automatic Storage Management, you must determine how many devices and the amount of free disk space that you require. To complete this task, follow these steps:

1. Determine whether you want to use Automatic Storage Management for Oracle datafiles, recovery files, or both.

   Note:

   You do not have to use the same storage mechanism for datafiles and recovery files. One can use the file system, while the other uses Automatic Storage Management. If you plan to use Automatic Storage Management for both datafiles and recovery files, then you should create separate ASM disk groups for the datafiles and the recovery files.

   If you plan to enable automated backups during the installation, then you can choose Automatic Storage Management as the storage mechanism for recovery files by specifying an ASM disk group for the flash recovery area. Depending how you choose to create a database during the installation, you have the following options:

   • If you select an installation method that runs Database Configuration Assistant in interactive mode (for example, by choosing the Advanced database configuration option), then you can decide whether you want to use the same ASM disk group for datafiles and recovery files. You can also choose to use different disk groups for each file type. Ideally, you should create separate ASM disk groups for datafiles and recovery files.

     The same choice is available to you if you use Database Configuration Assistant after the installation to create a database.

   • If you select an installation type that runs Database Configuration Assistant in non-interactive mode, then you must use the same Automatic Storage Management disk group for datafiles and recovery files.

2. Choose the Automatic Storage Management redundancy level that you want to use for the Automatic Storage Management disk group.

   The redundancy level that you choose for the Automatic Storage Management disk group determines how ASM mirrors files in the disk group, and determines the number of disks and amount of disk space that you require. The redundancy levels are as follows:

   • External redundancy

     An external redundancy disk group requires a minimum of one disk device. The effective disk space in an external redundancy disk group is the sum of the disk space in all of its devices.

     Because Automatic Storage Management does not mirror data in an external redundancy disk group, Oracle recommends that you use only RAID or similar devices that provide their own data protection mechanisms as disk devices in this type of disk group.

   • Normal redundancy

     In a normal redundancy disk group, Automatic Storage Management uses two-way mirroring by default (except for the control file, which is mirrored three ways), to increase performance and reliability. A normal redundancy disk group requires a minimum of two disk devices, or two failure groups. The effective disk space in a normal redundancy disk group is half the sum of the disk space in all of its devices.

     For most installations, Oracle recommends that you use normal redundancy disk groups.

   • High redundancy

     In a high redundancy disk group, Automatic Storage Management uses three-way mirroring to increase performance and provide the highest level of reliability. A high redundancy disk group requires a minimum of three disk devices (or three failure groups). The effective disk space in a high redundancy disk group is one-third the sum of the disk space in all of its devices.

     While high redundancy disk groups do provide a high level of data protection, you must consider the higher cost of additional storage devices before deciding to use this redundancy level.

3. Determine the total amount of disk space that you require for the datafiles and recovery files.

   Use the following table to determine the minimum number of disks and the minimum disk space requirements for the installation:

   Redundancy Level	Minimum Number of Disks	Datafiles	Recovery FIles	Both File Types
   External	1	1.15 GB	2.3 GB	3.45 GB
   Normal	2	2.3 GB	4.6 GB	6.9 GB
   High	3	3.45 GB	6.9 GB	10.35 GB

   
   

   If an ASM instance already exists on the system, then you can use an existing disk group to meet these storage requirements. If necessary, you can add disks to an existing disk group during the installation.

   The following section describes how to identify existing disk groups and determine the free disk space that they contain.

4. Optionally identify failure groups for the ASM disk group devices.

   Note:

   You need to complete this step only if you intend to use an installation method that runs Database Configuration Assistant in interactive mode–for example, if you intend to choose the Custom installation type, or the Advanced database configuration option. Other installation types do not enable you to specify failure groups.

   If you intend to use a normal or high redundancy disk group, then you can further protect your database against hardware failure by associating a set of disk devices in a custom failure group. Failure groups define ASM disks that share a common potential failure mechanism. For more information about ASM failure groups, refer to Oracle Database Administrator's Guide.

   Note:

   If you define custom failure groups, you must specify a minimum of two failure groups for normal redundancy disk groups and three failure groups for high redundancy disk groups.

5. If you are sure that a suitable disk group does not exist on the system, then install or identify appropriate disk devices to add to a new disk group. Use the following guidelines when identifying appropriate disk devices:

   • All of the devices in an Automatic Storage Management disk group should be the same size and have the same performance characteristics.

   • Do not specify two or more partitions on a single physical disk as ASM disks in the same disk group. Automatic Storage Management expects each device for a disk group to be on a separate physical disk.

   • Although you can specify a logical volume as a device in an Automatic Storage Management disk group, Oracle does not recommend their use. Logical volume managers can hide the physical disk architecture, preventing Automatic Storage Management from optimizing I/O across the physical devices.

   Tip:

   As you progress through the following steps, make a list of the raw device names you intend to use and have it available during your database or ASM installation.

Step 2 (Optional): Using an Existing Automatic Storage Management Disk Group

If you want to use Automatic Storage Management as the storage option for either database or recovery files, and an existing Automatic Storage Management disk group already exists, then you have the following choices, depending on the installation method that you select:

• If you select an installation method that runs Database Configuration Assistant in interactive mode (for example, by choosing the Advanced database configuration option for example), then you can decide whether you want to create a new disk group, or use an existing disk group.

  The same choice is available to you if you use Database Configuration Assistant after the installation to create a database.

• If you select an installation type that runs Database Configuration Assistant in non-interactive mode, then you must choose an existing disk group for the new database; you cannot create a new disk group. However, you can add disk devices to an existing disk group if the existing disk group has insufficient free space for your requirements.

To determine whether an existing ASM disk group exists, or to determine whether there is sufficient disk space in a disk group, you can use Oracle Enterprise Manager, either Grid Control or Database Control. Alternatively, you can use the following procedure:

1. In the Services Control Panel, make sure that the OracleASMService+ASMn service, where n is the node number, has started.

2. Open a Windows command prompt and temporarily set the ORACLE_SID environment variable to specify the appropriate value for the ASM instance that you want to use.

   For example, if the ASM SID is named +ASM1, then enter a setting similar to the following:

   C:\> set ORACLE_SID = +ASM1
   

3. Connect to the Automatic Storage Management instance as the SYS user with AS SYSDBA privilege and start the instance if necessary with a command similar to the following:

   C:\> sqlplus "SYS/SYS_password as SYSDBA"
   SQL> STARTUP
   

4. Enter the following command to view the existing disk groups, their redundancy level, and the amount of free disk space in each disk group:

   SQL> SELECT NAME,TYPE,TOTAL_MB,FREE_MB FROM V$ASM_DISKGROUP;
   

5. From the output, identify a disk group with the appropriate redundancy level and note the free space that it contains.

6. If necessary, install, or identify the additional disk devices required to meet the storage requirements listed in the previous section.

   Note:

   If you are adding devices to an existing disk group, then Oracle recommends that you use devices that have the same size and performance characteristics as the existing devices in that disk group.

Step 3: Creating DAS or SAN Disk Partitions for Automatic Storage Management

To use a DAS or SAN disk in Automatic Storage Management, the disk must have a partition table. Oracle recommends creating exactly one partition for each disk containing the entire disk.

Use Microsoft Computer Management or the command line tool diskpart to create the partitions. Ensure that you create the partitions without drive letters. After you have created the partitions, the disks can be configured.

Step 4: Manually Configuring Disks for Automatic Storage Management

To use Automatic Storage Management with direct attached storage (DAS) or storage area network (SAN) storage, the disks must be stamped with a header. If you install Oracle Database in interactive mode, then Oracle Universal Installer configures the disks' headers during the installation process.

However, if you plan to install Oracle Database in noninteractive mode, then you need to configure the disks manually before installation either by using asmtoolg (GUI version) or using asmtool (command line version). You can also use these tools to reconfigure the disks after installation. The asmtoolg and asmtool utilities only work on partitioned disks; you cannot use Automatic Storage Management on unpartitioned disks.

The following section describes the asmtoolg and asmtool functions and commands.

asmtool -add [-force]
\Device\Harddisk1\Partition1 ORCLDISKASM0
\Device\Harddisk2\Partition1 ORCLDISKASM2...

asmtool -addprefix ORCLDISKASM [-force]
\Device\Harddisk1\Partition1
\Device\Harddisk2\Partition1...

asmtool -list [-force]

asmtool -delete ORCLDISKASM0 ORCLDISKASM1...

Creating Partitions for Logical Volumes

Create the logical volumes listed in the following table. You must create these volumes to install an Oracle database.

To create and configure raw volumes or partitions, use the disk administration tools provided by the operating system or third party vendors. The following administration tools are provided by the operating system:

• Windows 2000 and Windows Server 2003 provide Disk Management snap-in to manage disks.

  To access this tool, type diskmgmt.msc at the command prompt. Alternatively, from the Start menu, select Programs, then Administrative Tools, then Computer Management. Then select the Disk Management node in the Storage tree.

• Windows Server 2003 provides a command line tool to manage disks.

  To access this tool, type diskpart.exe at the command prompt.

http://www.microsoft.com/

You can use the diskpart tool command create partition to create primary or extended partitions, or create logical drives.

The following example uses the diskpart tool to create a 32 MB extended partition on disk 100. In this syntax, diskpart.exe is the command line tool for managing disks.

c:\> diskpart.exe
DISKPART> select disk 100
DISKPART> create partition extended size=32

Assigning Logical Names

After creating volumes, assign logical names for the Oracle Database. You can assign names to partitions by using importSYMLinks from the command line, or by using Oracle Object Link Manager. To use Oracle Object Link Manager to create persistent symbolic links to the corresponding raw partitions, run the command CRS_home\bin\GUIOracleObjManager.exe

Creating the DBCA Raw Device Mapping File

To enable Database Configuration Assistant (DBCA) to identify the appropriate raw partition symbolic links for each database file, you must create a raw device mapping file, as follows:

1. Set the ORACLE_BASE environment variable to specify the Oracle base directory that you identified or created previously, as in this example:

   C:\>set ORACLE_BASE = E:\oracle
   

2. Create a database subdirectory under the Oracle base directory as in this example:

   C:\>mkdir E:\oracle\dbname
   

   where dbname is the name of the database that you chose previously.

3. Change directory to the %ORACLE_BASE%\dbname directory.

4. Using any text editor, create a file called conf.txt. The file should have the following characteristics:

   • Each line in the file must have the following format:

     database_object_identifier = symbolic link name
     

   • For your Oracle RAC database, the file must specify one automatic undo tablespace datafile (undotbsn) and two redo log files (redon_1, redon_2) for each instance where n is the instance number.

   • Specify at least two control files (control1, control2).

   Note:

   In Windows, by default, \ represents the escape key. To enter a backslash as part of a script, you must enter it in as a string literal. This means that when configuring the mapping file, for Windows to read the mapping file with the path \\.\, you must enter the path as \\\\.\\. Windows reads this as "escape backslash escape backslash. escape backslash."

   The following syntax example is for a mapping file for a two-instance Oracle RAC cluster:

   system=\\\\.\\dbname_SYSTEM 
   sysaux=\\\\.\\dbname_SYSAUX 
   spfile=\\\\.\\dbname_SPFILE 
   users=\\\\.\\dbname_USERS 
   temp=\\\\.\\dbname_TEMP 
   undotbs1=\\\\.\\dbname_UNDOTBS1 
   undotbs2=\\\\.\\dbname_UNDOTBS2 
   control1=\\\\.\\dbname_CONTROL1 
   control2=\\\\.\\dbname_CONTROL2 
   redo1_1=\\\\.\\dbname_REDO1_1 
   redo1_2=\\\\.\\dbname_REDO1_2 
   redo2_1=\\\\.\\dbname_REDO2_1 
   redo2_2=\\\\.\\dbname_REDO2_2 
   example=\\\\.\\dbname_EXAMPLE 
   pwdfile=\\\\.\\dbname_pwdfile
   

5. Save the file and note the file name that you specified.

6. You may optionally set an environment variable, DBCA_RAW_CONFIG, to specify the full path to this file. For the Oracle base defined in Step 1, you would use the following command:

   C:\>set DBCA_RAW_CONFIG=E:\oracle\dbname\conf.txt