Backups and UPS, the forgotten ones
I think anyone who has worked with an IT infrastructure for some time has ever encountered one of these problems:
- The backups had stopped months ago and nobody had noticed (because of a change of password, a change of location, filling up the target disk, failure of the NAS disks, etc.)
- UPS (Uninterruptible Power Supply) do not last a single sigh when the power goes out: after years without power cuts the batteries no longer work or the system has been charging uncontrollably until they are consumed immediately in the event of a power cut.
- The machines and equipment that we have in high availability and that must act when their "masters" fail are off, out of date, have a full hard drive, etc.
Luckily, as always, the monitoring system comes to the rescue. We use our platform, based on
Zabbix 5.0 and Grafana 7, but what we have here also works for other platforms, although it may be more complex or not as visual.
Monitoring of backups
There are many approaches to managing backups. From the typical script that runs in cron every night to platforms like
Veeam Backup.
For the first case and with most platforms, the usual thing is to finish the script or configure the tool to send an email with the result. This, if you have few servers or computers, is fine, typical mail to check with the coffee cup in hand. But as soon as you have a certain volume or you prefer to save 5 minutes every day, the best thing is to have a tool that sends you an email ONLY if it has failed. Besides, what happens when you don't get that email?
Has the email failed or hasn't been backed up? Who checks those emails when you're not there?
The approach to be applied is the same as for any monitoring, only to warn if there are problems. We can put our monitoring tool to read emails or if the script is ours, simply send a Zabbix trapper or via API a 1 or 0 to our monitoring platform. From it we can indicate how long to wait before giving an alarm if we do not get a 1 or a 0 in a certain time, etc. Total flexibility.
Veeam Backup works by email normally, but the truth is that reading emails with any platform, although feasible, often causes problems. Fortunately this platform also works with SNMP traps, warning us of the result of any work. We configure it immediately to send the traps to our Zabbix (more information at:
https://helpcenter.veeam.com/docs/one/alarms/snmp_traps.html?ver=100)