Here are 10 steps to move an Opsview master server and limit downtime during the relocation of your IT monitoring system.
Keep in mind in this scenario we have the luxury of moving the server while maintaining the same architecture and OS version, so we have the ability to copy files from the old server to the new server. We are not restoring the master server. (check out Opsview documentation on backups and restores to find out which databases and files are backed up by a nightly cron job).
The process can be broken down into three sections: components, configuration, and communication.
Components(For this section, stop all Opsview processes on the master and all slaves>
1. On the new server, follow the documentation to install Opsview for your Linux distribution. Also install MySQL and Apache in the same fashion as your original server, whether it is with a package manager or custom compile.
2. Opsview does daily backups that can be used for a restore. In this case, we are doing a migration so we can manually copy all the data required. We will treat Opsview like any other web application and rsync the entire /usr/local/nagios directory to the new server and do a mysqldump of all MySQL databases, capturing odw, opsview, reports, runtime, and the database called mysql with all users and their permissions.
nagios@< oldserver >$ rsync -arolptv /usr/local/nagios/
root@< oldserver ># mysqldump --routines -u’dbuser’ -p’dbpass’ --all-databases > /tmp/alldbs.dmp
3. Copy and import the MySQL databases on the new server. Also, verify the nagios user has the correct (and recursive) permissions on /usr/local/nagios.
4. Time is crucial between the master and slave servers and you will get an error if the time is only seconds apart. Be sure all slaves and the master server are using the same time server and they are synced before turning on Opsview for the first time.
Configuration(For this section, don’t start up Opsview yet)
5. On the slave servers, the master server is listed in /usr/local/nagios/etc/opsview-slave.conf either by IP address or fully qualified domain name. Change the IP address here or change /etc/hosts to have the new IP address and keep the FQDN in the configuration file. We want to control how the slaves access the master and not rely on any other DNS source.
6. If you use Apache as a proxy server, copy over /etc/httpd/conf.d/opsview.conf from the old server to the new server.
7. If you use the rsync method to copy all files as the nagios user, permissions on the file /usr/local/nagios/libexec/check_icmp (and check_dhcp) may be changed to an incorrect value. Set the correct ownership and access permissions.
chown root:nagios /usr/local/nagios/libexect/check_icmp chmod 4550 /usr/local/nagios/libexec/check_icmp
Communication(Wait for it SSH connections must work before Opsview can be started).
8. Since we have the old server, copy all the SSH keys from the nagios home directory (probably under /var/log/nagios/.ssh/) to the new server. On the slaves, remove the entry for the master in the known_hosts file under the nagios user and establish a connection to the new master server to verify SSH works correctly with shared keys.
9. Use the send2slaves command to test the master communicating with the slaves.
10. When you have all components installed and started, all configuration files in place, and communication established from master to slaves (and vice versa), start Opsview and Opsview-web on the master server.
/etc/init.d/opsview start /etc/init.d/opsview-web start Start Opsview on all the slave servers.
/etc/init.d/opsview-agent start /etc/init.d/opsview-slave start
Done! The Opsview UI should be accessible (after a DNS change or an edit to your local hosts file to verify). Moving the master server shouldn’t be a frequent task, but with Opsview’s portable architecture, it can easily be accomplished.
Want to learn more about configuring and implementing IT monitoring with Opsview? Check out our training courses.