This section provides information about Microsoft® Cluster Service (MSCS). This section is intended to be an overview of MSCS and provides information about the following:
Cluster objects
Cluster networks
Network interfaces
Cluster nodes
Groups
Cluster resources
File share resources
Failover and failback
For information about specific MSCS procedures, see the MSCS online help.
NOTE: In this guide and in other cluster documentation, the quorum resource is also referred to as the
quorum disk.
Cluster Objects
Cluster objects are the physical and logical units managed by MSCS. Each object is associated with the following:
One or more properties, or attributes, that define the object and its behavior within the cluster.
A set of cluster control codes used to manipulate the object's properties.
A set of object management functions used to manage the object through MSCS.
Cluster Networks
A network performs one of the following roles in a cluster:
A network that carries internal cluster communication
A public network that provides client systems with access to cluster application services
A public-and-private network that carries both internal cluster communication and connects client systems to cluster application services
Neither a public nor private network that carries traffic unrelated to cluster operation
Preventing Network Failure
MSCS uses all available private and public-and-private networks for internal communication. Configure multiple networks as private or public-and-private to protect the cluster from a single network failure. If there is only one such network available and it fails, the cluster nodes stop communicating with each other. When two nodes are unable to communicate, they are partitioned and MSCS automatically shuts down on one node. While this shutdown guarantees the consistency of application data and the cluster configuration, it can make cluster resources unavailable.
For example, if each node has only one network adapter, and the network cable on one of the nodes fails, each node, (because it is unable to communicate with the other), attempts to take control of the quorum disk. There is no guarantee that the node with a functioning network connection will gain control of the quorum disk. If the node with the failed network cable gains control, the entire cluster is unavailable to network clients. To avoid this problem, ensure that all nodes have at least two networks and are configured to use both networks for the private network (internal communications).
Node-to-Node Communication
MSCS does not use public only networks for internal communication. For example, a cluster has Network A configured as private and Network B configured as public. If Network A fails, MSCS does not use Network B because it is public; the nodes stop communicating and one node terminates its Cluster Service.
Network Interfaces
The Microsoft® Windows® operating system keeps track of all network adapters in a server cluster. This tracking system allows you to view the state of all cluster network interfaces from a cluster management application, such as Cluster Administrator.
Cluster Nodes
A cluster node is a system in a server cluster that has a working installation of the Windows operating system and MSCS.
Cluster nodes have the following characteristics:
Every node is attached to one or more cluster storage devices. Each cluster storage device attaches to one or more disks. The disks store all of the cluster's configuration and resource data. Each disk can be owned by only one node at any point in time, but ownership can be transferred between nodes. The result is that each node has access to all cluster configuration data.
Every node communicates with the other nodes in the cluster through one or more network adapters that attach nodes to networks.
Every node in the cluster is aware of another system joining or leaving the cluster.
Every node in the cluster is aware of the resources that are running on all nodes in the cluster.
All nodes in the cluster are grouped under a common cluster name, which is used when accessing and managing the cluster.
Table 5-1 defines various states of a node that can occur in cluster operation.
Table 5-1. Node States and Definitions
State
Definition
Down
The node is not actively participating in cluster operations.
Joining
The node is in the process of becoming an active participant in the cluster operations.
Paused
The node is actively participating in cluster operations but cannot take ownership of resource groups and cannot bring resources online.
Up
The node is actively participating in all cluster operations, including hosting cluster groups.
Unknown
The state cannot be determined.
When MSCS is installed for the first time on a node, the administrator must choose whether that node forms its own cluster or joins an existing cluster. When MSCS is started on a node, that node searches for other active nodes on networks enabled for internal cluster communications.
Forming a New Cluster
If a cluster cannot be joined, the node attempts to form the cluster by gaining control of the quorum disk. If the node gains control of the quorum disk, the node forms the cluster and uses the recovery logs in the quorum disk to update its cluster database. MSCS maintains a consistent, updated copy of the cluster database on all active nodes.
Joining an Existing Cluster
A node can join an existing cluster if it can communicate with another cluster node. If a cluster exists and the joining node finds an active node, it attempts to join that node's cluster. If it succeeds, MSCS then validates the node's name and verifies version compatibility. If the validation process succeeds, the node joins the cluster. The node is updated with the latest copy of the cluster database.
Groups
A group is a collection of cluster resources with the following characteristics:
All of the resources in the group are moved to the alternate node when one resource in a group fails and it is necessary to move the resource to an alternate node.
A group is always owned by one node at any point in time, and a resource is always a member of a single group. Therefore, all of a group's resources reside on the same node.
Groups enable resources to be combined into larger logical units. Typically a group is made up of related or dependent resources, such as applications and their associated peripherals and data. However, groups can also be established with resources that are unrelated and nondependent to balance the load or for administrative convenience.
Every group maintains a prioritized list of the nodes that can and should act as its host. The preferred nodes list is generated by MSCS. Cluster Service produces a list of preferred nodes for a group from the list of possible owners that is maintained by the group's resources and can be modified by an Administrator.
To maximize the processing power of a cluster, establish at least as many groups as there are nodes in the cluster.
Cluster Resources
A cluster resource is any physical or logical component that has the following characteristics:
Can be brought online and taken offline
Can be managed in a server cluster
Can be hosted (owned) by only one node at a time
To manage resources, MSCS communicates to a resource DLL through a Resource Monitor. When MSCS makes a request of a resource, the Resource Monitor calls the appropriate entry-point function in the resource DLL to check and control the resource's state.
Dependent Resources
A dependent resource requires another resource to operate. For example, a network name must be associated with an IP address. Because of this requirement, a network name resource is dependent on an IP address resource. A resource can specify one or more resources on which it is dependent. A resource can also specify a list of nodes on which it is able to run. Preferred nodes and dependencies are important considerations when administrators organize resources into groups.
Dependent resources are taken offline before the resources upon which they depend are taken offline, likewise, they are brought online after the resources on which they depend are brought online.
Setting Resource Properties
Using the resource Properties dialog box, you can perform the following tasks:
View or change the resource name
View or change the resource description and possible owners
Assign a separate memory space for the resource
View the resource type, group ownership, and resource state
View which node currently owns the resource
View pre-existing dependencies and modify resource dependencies
Specify whether to restart a resource and the settings used to restart the resource (if required)
Check the online state of the resource by configuring the Looks Alive and Is Alive polling intervals in MSCS
Specify the time requirement for resolving a resource in a pending state (OnlinePending or Offline Pending) before MSCS places the resource in Offline or Failed status
Set specific resource parameters
The General, Dependencies, and Advanced tabs are the same for every resource. Some resource types support additional tabs.
Properties of a cluster object should not be updated on multiple nodes simultaneously. See the MSCS online documentation for more information.
Resource Dependencies
Groups function properly only if resource dependencies are configured correctly. MSCS uses the dependencies list when bringing resources online and offline. For example, if a group in which a physical disk and a file share are located is brought online, the physical disk containing the file share must be brought online before the file share.
Table 5-2 shows resources and their dependencies. The resources in the right column must be configured before you create the resource.
Table 5-2. Cluster Resources and Required Dependencies
Resource
Required Dependencies
File share
Network name (only if configured as a distributed file system [DFS] root)
IP address
None
Network name
IP address that corresponds to the network name
Physical disk
None
Setting Advanced Resource Properties
You can configure the advanced resource properties using the Advanced tab in the resource Properties dialog box. Use the Advanced tab to have MSCS perform the following tasks:
Restart a resource or allow the resource to fail.
To restart the resource, select Affect the group (if applicable).
To fail over the resource group to another cluster node when the resource fails, select Affect the group and then enter the appropriate values in Threshold and Period. If you do not select Affect the group, the resource group will not fail over to the healthy cluster node.
The Threshold value determines the number of attempts by MSCS to restart the resource before the resource fails over to a healthy cluster node.
The Period value assigns a time requirement for the Threshold value to restart the resource.
Adjust the time parameters for Looks Alive (general check of the resource) or Is Alive (detailed check of the resource) to determine if the resource is in the online state.
Select the default number for the resource type.
To apply default number, select Use resource type value.
Specify the time parameter for a resource in a pending state (Online Pending or Offline Pending) to resolve its status before moving the resource to Offline or Failed status.
Resource Parameters
The Parameters tab in the Properties dialog box is available for most resources. Table 5-3 lists each resource and its configurable parameters.
Table 5-3. Resources and Configurable Parameters
Resource
Configurable Parameters
File share
Share permissions and number of simultaneous users
Share name (clients will detect the name in their browse or explore lists)
Share comment
Shared file path
IP address
IP address
Subnet mask
Network parameters for the IP address resource (specify the correct cluster network)
Network name
System name
Physical disk
Drive for the physical disk resource (the drive cannot be changed after the resource is created)
Quorum Disk (Quorum Resource)
The quorum resource is a common resource in the cluster that is accessible by all of the cluster nodes. Normally a physical disk on the shared storage, the quorum resource maintains data integrity, cluster unity, and cluster operations—such as forming or joining a cluster—by performing the following tasks:
Enables a single node to gain and defend its physical control of the quorum resource — When the cluster is formed or when the cluster nodes fail to communicate, the quorum resource guarantees that only one set of active, communicating nodes is allowed to form a cluster.
Maintains cluster unity —The quorum resource allows cluster nodes that can communicate with the node containing the quorum resource to remain in the cluster. If a cluster node fails for any reason and the cluster node containing the quorum resource is unable to communicate with the remaining nodes in the cluster, MSCS automatically shuts down the node that does not control the quorum resource.
Stores the most current version of the cluster configuration database and state data —If a cluster node fails, the configuration database helps the cluster recover a failed resource or recreate the cluster in its current configuration.
The only type of resource supported by MSCS that can act as a quorum resource is the physical disk resource. However, developers can create their own quorum disk types for any resources that meet the arbitration and storage requirements.
Using the Quorum Disk for Cluster Integrity
The quorum disk is also used to ensure cluster integrity by performing the following functions:
Maintaining the cluster node database
Ensuring cluster unity
When a node joins or forms a cluster, MSCS must update the node's private copy of the cluster database. When a node joins an existing cluster, MSCS can retrieve the data from the other active nodes. However, when a node forms a cluster, no other node is available. MSCS uses the quorum disk's recovery logs to update the node's cluster database, thereby maintaining the correct version of the cluster database and ensuring that the cluster is intact.
For example, if node 1 fails, node 2 continues to operate, writing changes to the cluster database. Before you can restart node 1, node 2 fails. When node 1 becomes active, it updates its private copy of the cluster database with the changes made by node 2 using the quorum disk's recovery logs to perform the update.
To ensure cluster unity, the operating system uses the quorum disk to ensure that only one set of active, communicating nodes is allowed to operate as a cluster. A node can form a cluster only if it can gain control of the quorum disk. A node can join a cluster or remain in an existing cluster only if it can communicate with the node that controls the quorum disk.
For example, if the private network (cluster interconnect) between cluster nodes 1 and 2 fails, each node assumes that the other node has failed, causing both nodes to continue operating as the cluster. If both nodes were allowed to operate as the cluster, the result would be two separate clusters using the same cluster name and competing for the same resources. To solve this problem, MSCS uses the node that owns the quorum disk to maintain cluster unity and solve this problem. In this scenario, the node that gains control of the quorum disk is allowed to form a cluster, and the other fails over its resources and becomes inactive.
Resource Failure
A failed resource is not operational on the current host node. At periodic intervals, MSCS checks to see if the resource appears operational by periodically invoking the Resource Monitor. The Resource Monitor uses the resource DLL for each resource to detect if the resource is functioning properly. The resource DLL communicates the results back through the Resource Monitor to MSCS.
Adjusting the Poll Intervals
You can specify how frequently MSCS checks for failed resources by setting the Looks Alive (general resource check) and Is Alive (detailed resource check) poll intervals. MSCS requests a more thorough check of the resource's state at each Is Alive interval than it does at each Looks Alive interval; therefore, the Is Alive poll interval is typically longer than the Looks Alive poll interval.
NOTE: Do not adjust the Looks Alive and Is Alive settings unless instructed by technical support.
Adjusting the Threshold and Period Values
If the resource DLL reports that the resource is not operational, MSCS attempts to restart the resource. You can specify the number of times MSCS can attempt to restart a resource in a given time interval. If MSCS exceeds the maximum number of restart attempts (Threshold value) within the specified time period (Period value), and the resource is still not operational, MSCS considers the resource to be failed.
NOTE: See "Setting Advanced Resource Properties" to configure the Looks alive, Is alive, Threshold,
and Period values for a particular resource.
NOTE: Do not adjust the Threshold and Period values settings unless instructed by technical support.
Configuring Failover
You can configure a resource to fail over an entire group to another node when a resource in that group fails for any reason. If the failed resource is configured to cause the group that contains the resource to fail over to another node, Cluster Service will attempt a failover. If the number of failover attempts exceeds the group's threshold and the resource is still in a failed state, MSCS will attempt to restart the resource. The restart attempt will be made after a period of time specified by the resource's Retry Period On Failure property, a property common to all resources.
When you configure the Retry Period On Failure properly, consider the following guidelines:
Select a unit value of minutes, rather than milliseconds (the default value is milliseconds).
Select a value that is greater or equal to the value of the resource's restart period property. This rule is enforced by MSCS.
NOTE: Do not adjust the Retry Period On Failure settings unless instructed by technical support.
Resource Dependencies
A dependent resource requires—or depends on—another resource to operate. For example, if a Generic Application resource requires access to clustered physical storage, it would depend on a physical disk resource.
The following terms describe resources in a dependency relationship:
Dependent resource — A resource that depends on other resources (the dependencies).
Dependency — A resource on which another resource depends.
Dependency tree — A series of dependency relationships such that resource A depends on resource B, resource B depends on resource C, and so on.
Resources in a dependency tree obey the following rules:
A dependent resource and all of its dependencies must be in the same group.
MSCS takes a dependent resource offline before any of its dependencies are taken offline, and brings a dependent resource online after all its dependencies are online, as determined by the dependency hierarchy.
Creating a New Resource
Before you add a resource to your NAS SCSI cluster, you must verify that the following elements exist in your cluster:
The type of resource is either one of the basic types provided with MSCS or a custom resource type provided by the application vendor, Microsoft, or a third party vendor.
A group that contains the resource already exists within your cluster.
All dependent resources have been created.
A separate Resource Monitor—recommended for any resource that has caused problems in the past.
To create a new resource:
Click the Start button and select Programs→ Administrative Tools→ Cluster Administrator.
The Cluster Administrator window appears.
In the console tree (usually the left pane), double-click the Groups folder.
In the details pane (usually the right pane), click the group to which you want the resource to
belong.
On the File menu, point to New, and then click Resource.
In the New Resource wizard, type the appropriate information in Name and Description, and
click the appropriate information in Resource type and Group.
Click Next.
Add or remove possible owners of the resource, and then click Next.
The New Resource window appears with Available resources and Resource dependencies selections.
To add dependencies, under Available resources, click a resource, and then click Add.
To remove dependencies, under Resource dependencies, click a resource, and then click
Remove.
Repeat step 7 for any other resource dependencies, and then click Finish.
Set the resource properties.
For more information on setting resource properties, see the MSCS online help.
Deleting a Resource
Click the Start button and select Programs→ Administrative Tools→ Cluster Administrator.
The Cluster Administrator window appears.
In the console tree (usually the left pane), click the Resources folder.
In the details pane (usually the right pane), click the resource you want to remove.
In the File menu, click Delete.
When you delete a resource, Cluster Administrator also deletes all the resources that have a dependency on the deleted resource.
File Share Resources
Creating a Cluster-Managed File Share
Launch Windows Explorer.
On a shared volume, create a new folder for the file share.
NOTE: Do not create a share for this folder.
Right-click the folder and select Properties.
In the Properties window, click the Security tab.
In the Group or users names box, verify that the Cluster Service account has Full Control
rights to this folder for the NTFS file system.
Close Windows Explorer.
Click the Start button and select Programs→ Administrative→ Tools→ Cluster
Administrator.
In the Cluster Administrator left window pane, ensure that a physical disk resource exists in
the cluster.
In the Cluster Administrator left or right window pane, right-click and select New→
Resource.
In the New Resource window, perform the following steps:
In the Name field, type a name for the new share.
In the Description field, type a description of the new share (if required).
In the Resource type drop-down menu, select File Share.
In the Group drop-down menu, select the appropriate virtual server for your file share.
Click Next.
The Possible Owners window appears.
Select the appropriate cluster node(s) in the Available nodes box on which this resource can
be brought online.
Click the Add button to move the cluster node(s) to the Possible owners menu.
Click Next.
The Dependencies window appears.
In the Available resources menu, select the appropriate resource dependencies which must be
brought online first by the Cluster Service.
Click the Add button to move the resources to the Resource dependencies menu.
Click Next.
The File Share Parameters window appears.
Perform the following steps:
In the Share name field, type the name of the file share.
In the Path field, type the path to the file share.
In the Comment field, enter any additional information about the file share (if required).
Click Permissions and apply the appropriate group or user names and permissions for the
file share (if required), and then click OK.
Click Advanced and select the appropriate file share properties (if required), and then
click OK.
In the right window pane, right-click the share and select Bring Online.
Deleting a File Share
Click the Start button and select Programs→ Administrative→ Tools→ Cluster
Administrator.
In the Cluster Administrator window console tree, click the Resources folder.
In the right window pane, right-click the file share you want to remove and select Delete.
NOTE: When you delete a resource, Cluster Administratorautomatically deletes all the resources
that have a a dependency on the deleted resource.
DFS File Shares
You can use the File Share resource type selection in Cluster Administrator to create a resource that manages a stand-alone DFS root; however, fault-tolerant DFS roots cannot be managed by this resource. The DFS root File Share resource has required dependencies on a network name and an IP address. The network name can be either the cluster name or any other network name for a virtual server.
A cluster-managed DFS root is different from an Active Directory (or domain-based) DFS root. If the data set does not change very often, using and replicating a domain-based DFS root can be a better selection than a cluster-managed DFS root for providing high availability. If the data set changes frequently, replication is not recommended, and a cluster-managed DFS root is the better solution.
Table 5-4 provides a summary for choosing the appropriate DFS root management scheme.
See the Dell PowerVault 77xN NAS Systems Administrator's Guide for more information.
Table 5-4. Selecting the Appropriate DFS Root Management Scheme
Data Set Activity
DFS Root Management
Data changes often
Domain-based
Data does not change very often
Cluster-managed
NOTE: Microsoft Windows Storage Server 2003, Enterprise Edition supports multiple stand-alone DFS
roots. The DFS roots can exist in multiple resource groups and each group can be hosted on a different
node in the cluster.
File Share Resource Types
If you want to use a PowerVault NAS SCSI cluster as a high-availability file server, you will need to select the type of file share for your resource. Three ways to use this resource type are available:
Basic file share — Publishes a single file folder to the network under a single name.
Share subdirectories — Publishes several network names—one for each file folder and all of its immediate subfolders. This method is an efficient way to create large numbers of related file shares on a single file server.
For example, you can create a file share for each user with files on the cluster node.
DFS root — Creates a resource that manages a stand-alone DFS root. Fault tolerant DFS roots cannot be managed by this resource. A DFS root file share resource has required dependencies on a network name and an IP address. The network name can be either the cluster name or any other network name for a virtual server.
Enabling Cluster NFS File Share Capabilities
After you add a node to the cluster, enable the NFS file sharing capabilities by performing the following steps.
NOTE: Perform this procedure on one cluster node after you configure the cluster.
Open a command prompt.
At the prompt, type:
c:\dell\util\cluster
In the cluster directory, run the NFSShareEnable.bat file.
Failover and Failback
This section provides information about the failover and failback capabilities of MSCS.
Failover
When an individual NAS cluster resource fails on a cluster node, MSCS detects the resource failure and tries to restart the resource on the cluster node. If the restart attempt reaches a preset threshold, MSCS brings the running resource offline, moves the dependent resources to another cluster node, and restarts all resources and related dependencies on the other cluster node(s). This process of automatically moving resources from a failed cluster node to other healthy cluster node(s) is called failover.
To fail over and fail back running NAS cluster resources, the resources are placed together in a group so that MSCS can move the cluster resources as a combined unit. This process ensures that the failover and/or failback procedures transfers all of the user resources as transparently as possible.
After failover, the Cluster Administrator can reset the following recovery policies:
NAS cluster resource dependencies
NAS cluster resource(s) restart on the same cluster node
Workload rebalancing (or failback) when a failed cluster node is repaired and brought back online
Failover Process
MSCS attempts to fail over a group when any of the following conditions occur:
The node currently hosting the group becomes inactive for any reason.
One of the resources within the group fails, and it is configured to affect the group.
Failover is forced by the System Administrator.
When a failover occurs, MSCS attempts to perform the following procedures:
The group's resources are taken offline.
The resources in the group are taken offline by MSCS in the order determined by the group's dependency hierarchy: dependent resources first, followed by the resources on which they depend.
For example, if an application depends on a Physical Disk resource, MSCS takes the application offline first, allowing the application to write changes to the disk before the disk is taken offline.
The resource is taken offline.
Cluster Service takes a resource offline by invoking, through the Resource Monitor, the resource DLL that manages the resource. If the resource does not shut down within a specified time limit, MSCS forces the resource to shut down.
The group is transferred to the next preferred host node.
When all of the resources are offline, MSCS attempts to transfer the group to the node that is listed next on the group's list of preferred host nodes.
For example, if cluster node 1 fails, MSCS moves the resources to the next cluster node number, which is cluster node 2.
The group's resources are brought back online.
If MSCS successfully moves the group to another node, it tries to bring all of the group's resources online. Failover is complete when all of the group's resources are online on the new node.
MSCS continues to try and fail over a group until it succeeds or until the number of attempts occurs within a predetermined time span. A group's failover policy specifies the maximum number of failover attempts that can occur in an interval of time. MSCS will discontinue the failover process when it exceeds the number of attempts in the group's failover policy.
Modifying the Failover Policy
Because a group's failover policy provides a framework for the failover process, ensure that your failover policy is appropriate for your particular needs. When you modify your failover policy, consider the following guidelines:
Define the method in which MSCS detects and responds to individual resource failures in a group.
Establish dependency relationships between the cluster resources to control the order in which MSCS takes resources offline.
Specify Time-out, failover Threshold, and failover Period for your cluster resources
Time-out controls how long MSCS waits for the resource to shut down.
Threshold and Period control how many times MSCS attempts to fail over a resource in a particular period of time.
Specify a Possible owner list for your cluster resources. The Possible owner list for a resource controls which cluster nodes are allowed to host the resource.
Failback
When the System Administrator repairs and restarts the failed cluster node, the opposite process occurs. After the original cluster node has been restarted and rejoins the cluster, MSCS will bring the running application and its resources offline, move them from the failover cluster node to the original cluster node, and then restart the application. This process of returning the resources back to their original cluster node is called failback.
You can configure failback to occur immediately at any given time, or not at all. However, ensure that you configure the failback time during your offpeak hours to minimize the effect on users, as they may experience a delay in service until the resources come back online.
