SERVERware 3 Deployment Guide for Cluster Edition

From Bicom Systems Wiki


SERVERware Cluster Edition is a full featured product, utilizing all the latest technologies for resiliency and scalability which is achieved through storage mirroring and dedicated processing hosts. The following guide describes minimal and recommended hardware requirements as well as procedures for a successful deployment of SERVERware 3 - Cluster Edition.

Hardware Requirements

IMPORTANT: Please confirm the specs with your Account manager before acquiring the hardware for the build.

To successfully install and operate the Mirror Edition, there are number of hardware requirements that need to be fulfilled.

For Cluster Edition, two Storage/Controller servers are needed and requirements per server are:

CPU Dual Quad-Core Intel Xeon Processor with 8MB Cache Dual Octa-Core Intel Xeon Processor with 12MB Cache
Ethernet 8 x 1Gb/s network interfaces 2 x 1Gb/s + 4 x 10Gb/s network interfaces
Disk 2 x 64GB Solid-State Drive ( for system ), 2 x 500GB Solid-State Drive ( for storage ) 2 x 64GB Solid-State Drive ( for system ), 4 x 1T Solid-State Drive ( for storage )

NOTE: 10Gbps ports are used for RAN and SAN connection for Storage hosts. It must be mentioned that if you want additional network redundancy as described in our Redundant network setup, four 10Gbps interfaces are recommended. As maximum number of storage drives PER SERVER should be: 8x drives for storage, 1x spare drive, 1x system drive. If we increase the number of drives in ZFS storage pool above 8x per server, this will lead to slower performance of the pool. Additionally, the 4 x 1T SSD drives for storage are a reference only, the actual size of the SSD drives depends on your needs.

Recommendation: It is better to use fewer larger drives than more smaller drives.

To effectively offload processing from Storage/Controller, at least 2 Processing Hosts must be added to the cluster. Although in Cluster Edition VPSes could run on the Storage servers, this is done only in a highly unlikely scenario where all Processing hosts are down, or the current Processing power is fully used and new servers need to be added later on. Processing Hosts should satisfy the following requirements:

CPU Dual Quad-Core Intel Xeon Processor with 8MB Cache Dual Octa-Core Intel Xeon Processor with 12MB Cache
Ethernet 4 x 1Gb/s network interfaces 2 x 1Gb/s + 2 x 10Gb/s network interfaces
Disk 2 x 64GB Solid-State Drive ( for system ) 2 x 64GB Solid-State Drive ( for system )

If used, Backup Hosts should satisfy the following requirements:

CPU 2.0 GHz Quad-Core Intel Processor with 8MB Cache 2.4 GHz Quad-Core Intel Xeon Processor with 12MB Cache
Ethernet 2 x 1Gb/s network interfaces 2 x 1Gb/s + 2 x 10Gb/s network interfaces
Disk 2 x 1TB Hard Disk Drive ( for system and backup ) 2 x 64GB Solid-State Drive ( for system ), 6 x 2TB Hard Disk Drive ( for backup )

IMPORTANT NOTE: Software RAID (including motherboard) implementations are not supported and could cause potential problems that Bicom Systems would not be able to support.

If physical access to the server is not possible, KVMoIP (Keyboard, Video, Mouse over IP) remote access to each server should provide:

  • Remote power management support (remote reset/power off/on)
  • Remote access to BIOS
  • Remote SSH console access
  • A public IP assigned to KVMoIP
  • If KVMoIP is behind firewall/NAT, the ports that need to be opened depend on the vendor.

For Example:

  • HP iLO: TCP/ 22, 23, 80, 443, 3389, 17988, 9300, 17990, 3002
  • SuperMicro: TCP/ 80, 332, 5901, 5900, 5120, 5123 UDP/ 623
  • Dell iDrac: TCP/ 22, 23, 80, 443, 161, 3668, 5869, 5900, 5901

But, we recommend checking the manufacturer documentation for more accurate information.

SERVERware installation media is required in one of the following forms:

  • DVD image burned to a DVD and inserted into DVD drive
  • USB image burned to a 2GB USB drive and inserted into operational USB port
  • Most KVMoIP solutions offer the ability to boot and install systems from remote locations by providing an URL to the desired install image.

Also, additional securities in operating your servers should be observed:

Deployment Guide

SERVERware 3 Cluster Edition topology is shown on the following image:

Cluster Edition topology layout

Typical Cluster Edition setup consists of two Storage servers, two Processing servers and optionally a backup server. It is highly recommended for storage servers to have identical number of disks with identical size. Although servers where this cannot be true can be made connected in a mirror, underlying disks management will pair up disks in such a way to maximize disk usage. Nevertheless, some disk space will be unusable.

NOTE: As can be seen, Processing servers do not need multiple disks for storage purposes, since all data is located on the Storage servers.

IMPORTANT: It is highly recommended that servers used for Storage are closely matched as different configurations, not to mention, using different manufacturers, are unlikely to work. Specifically, problems that may arise are network interface naming conventions that are different on various motherboards/network controllers. This of course does not affect Processing servers which can be varied.

Network setup

In Cluster Edition there is a need for three different networks:

  • LAN network which is used to transport SIP traffic and access storage and optional backup server.
  • RAN network which is used exclusively between storage servers for replication purposes
  • SAN network used to connect Processing servers with Storage servers, and through VPSes from Storage are executed on Processing hosts.

It is recommended that SAN and RAN networks are isolated from LAN network and accessible only from respective servers using them.

How-to instructions regarding redundant network setup are available on the following page:

Disk setup

Because of ZFS features, hardware RAID is not required. If there is a RAID controller present and you want to use it, working battery and write cache are a must. Any hardware RAID without battery backup and write cache enabled can't be supported. Fake RAID is not supported, should not be used and must be turned off if supported by motherboard (Fake RAID is essentially software RAID provided by the BIOS on the motherboard). More on Hardware RAID controllers AND OpenZFS

Lets say we have two disks for storage per server (it is highly recommended that disks should be identical).

Currently, setup wizard will form the following pool:

NETSTOR pool without hardware RAID

As you can see from the following image, the system will take either a disk or logical volume detected on server and form a ZFS mirror through RAN between the two storage servers.

Creation of USB installation media

USB images and instructions are available on the following how-to page:

Installation wizard steps

To install the SERVERware onto the designated server, system administrator must either connect a prepared USB stick, or use the KVM ability to remote boot installation image given as remote URL. Whether the administrator uses the USB installation image or remote boots image, first part of this procedure will be CLI based and the welcome screen:

sw3 deploy dl360 g7 1.png

If the live system was able to pickup IP address from DHCP server, it will show so on this screen. You can then access the system remotely via ssh on port 2020 with username 'root' and password 'bicomsystems' and continue installation. There are several options offered on the welcome screen:

  • Exit - Choose this option to exit installation wizard. This option will open live system command line shell.
  • Verify Media - This option will go trough the installation media files, match them against previous stored checksum and check for corruption.
  • Next - Proceed to the next step.
  • Network - Configure IP address for the remote access to the installation wizard.

Step 1:

Select the type of installation you want to install, Storage/Controller or Host (Processing or Backup).

sw3 deploy dl360 g7 2.png

Storage/Controller is the network storage for VPSs on the SERVERware network. In order to use mirrored setup, you will have to install two physical machines as Storage/Controller. Processing Host is computation resource that will attach and execute VPSs from storage via SAN (Storage Area Network).

Step 2:

Installation wizard will proceed to check for available disks.

sw3 deploy dl360 g7 3.png

Step 3:

Select physical disk for system installation. Volume for storage will be automatically created from disks that are not selected in this step.

sw3 deploy dl360 g7 4.png

Step 4:

Confirmation dialog appears.

sw3 deploy dl360 g7 5.png

Step 5:

Installation wizard will now proceed with installation of SERVERware operating system.

sw3 deploy dl360 g7 6.png
sw3 deploy dl360 g7 7.png

Step 6:

After OS is installed, configure network dialog appears.

sw3 deploy dl360 g7 8.png

Click Next to continue and then chose one of the options to configure network interface.

sw3 deploy dl360 g7 13b.png

After finishing network configuration, click next to finish installation. Wizard will initiate reboot.

Step 7:

Redo installation steps for the second (mirrored) machine.

Setup wizard steps

Open your browser and enter LAN IP you configured upon installation. After confirming self signed certificate, SERVERware setup wizard login screen appears. Enter administration password which is by default 'serverware'.

sw3 deploy dl360 g7 14.png

Step 1:

After successful login, SERVERware EULA appears.

sw3 deploy dl360 g7 15.png

Acceptance of the EULA leads to the next step.

Step 2:

Enter your license number, administrator's email, set new administrator's SERVERware GUI password. This password will also apply to shell root account. Select your time zone and click next to continue.

sw3 deploy dl360 g7 16.png

Step 3:

Depending on the license you acquired, this step will offer you to configure LAN, SAN and mirror RAN network. LAN is a local network for SERVERware management and service provision. SAN is a network dedicated to connecting SERVERware storage with processing hosts. RAN is a network dedicated to real time mirroring of two servers.

sw3 deploy dl360 g7 17.png

Setup wizard will suggest default configuration for the network interfaces. The machines must be on same LAN network and the same SAN and RAN network. Modify network configuration if needed and click Next to proceed to the next step.

Step 4:

Choose the name for the cluster if you don't like the one generated by the SERVERware (it must be valid per hostname constrains).

sw3 deploy dl360 g7 18.png

Select from the list or enter the LAN IP address of the second (mirrored) machine. Purpose of mirrored setup is to provide storage redundancy, so it will need a few more configuration parameters. LAN Virtual IP is a floating IP address for accessing mirrored storage server. SAN Virtual IP is floating IP address used for access to the storage. Administration UI IP address will be used for CONTROLLER VPS (GUI).

CONTROLLER VPS is setup automatically on the storage server, and its purpose is to provide administrative managing web console as well as to control and monitor SERVERware hosts and VPSs.

sw3 deploy dl360 g7 19.png

Once you click Finish button, wizard will initialise and setup the network storage. When complete, setup will present the summary info.

sw3 deploy dl360 g7 20.png

Wait a few seconds for CONTROLLER to start and click on the Controller Web Console link to start using SERVERware and creating VPSs.