Configuration

The inventory may define these hosts:

• observers: Set of hosts to observe the cluster (only the first is considering)

• Hypervisors: Set of machines to hosts VMs.

Node redundancy

All nodes in the cluster have access to a shared storage via Ceph (see Shared storage section). With it, the cluster is in N to N redundancy mode.

• Corosync will provides messaging and membership services.

• Pacemaker will manage the cluster (synchronize resources between each node).

...

More details on Pacemaker here and Corosync here.

pacemaker-remote

pacemaker-remote is a component which can be installed inside a VM to allow Pacemaker to manage and monitor resources inside this VM.

For instance, with pacemaker-remote pacemaker can monitor services and containers directly inside a VM.

...

...

...

...

For more information about pacemaker-remote refer to https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Remote/singlehtml/.

Management tool

The vm_manager project is an a high-level interface of pacemaker and ceph Ceph to manage the VM like a resource. He is installed during the installation step and provides the vm-mgr command.

Sub-command

All sub-commands has -n, --name required option to specify which resource should be used.

• add_colocation: Adds a colocation constraints between resources

• clone: Creates a copy of the VM

• create: Generates a new resource from a VM

• create_snapshot: Creates a snapshot of a resource

• disable: Stops and removes the resource on the cluster

• enable: Adds and starts the resource on the cluster

• get_metadata: Gets a metadata of a resource

• list: Lists all resources

• list_metadata: Lists all keys of a resource

• list_snapshots: Lists all created snapshots

• purge: Deletes all snapshots of a resource

• remove: Removes the resource

• remove_snapshot: Removes a snapshot of a resource

• rollback: Rollbacks to a snapshot for a resource

• set_metadata: Sets a metadata of a resource

• start: Start a resource

• status: Gets the status of a resource

• stop: Stops a resource

Resources status

• Undefined:
• Disabled:
• Failed:
• Started:
• Starting:
• Stopping:
• Stopped:

...

Replace a dead node

Please refer to How to manage VM in SEAPATH for more detais.

Node replacement

In case one of the nodes suffers a difficult to repair situation (lost motherboard for example, or lost disk with no RAID), it might become necessary to replace the server with a blank one.

From the cluster point of view, we will need to remove the old node and add the new one, for both corosync/pacemaker and ceph.

1. The ansible/playbooks/replace_machine_remove_machine_from_cluster.yaml playbook can remove a node

...

1. from the cluster. For this, the machine_to_remove should be set to the hostname to remove.
   The below command should be

...

1. launched in the ansible project.

   Code Block
   language bash
   cqfd run ansible-playbook -i /path/to/inventory.yaml -e machine_to_remove=HOSTNAME playbooks/replace_machine_remove_machine_from_cluster.yaml

2. A new

...

1. host should be

...

1. installed with the ISO installer and

...

1. the same hostname, ip address, etc... than the old node.

2. Make the "cluster network" connections between hosts.

3. Restart the cluster_setup_debian.yml playbook to configure the new host in the cluster (more details here).

...