News and Events

News and Events

Qflex v4.6.3 – Keep Alerting Even When the Monitoring Host Itself Is in Trouble

QFlex Standard 4.6.3 hardens the alerting pipeline so a failure in the very infrastructure QFlex runs on, such as a full disk or an unreachable database, no longer silences your alerts. This release reworks the alert notification sequence, adds a lightweight, database-independent monitoring watchdog, and refreshes the underlying platform.

 

Highlights

Alerts fire even when database writes fail. The alert email now goes out before QFlex tries to persist the alert record, so a failed write, for example a host that has run out of disk, can no longer suppress the alarm.

New monitoring watchdog. A small, self-contained safety net that independently checks that the database is reachable and the disks are healthy, and emails you if the normal, database-driven alert path has gone silent.

Platform refresh. Upgraded to Apache Tomcat 9.0.117, MySQL Connector/J 8.0.33, a sensible default JVM heap (1 GB to 2 GB), and a Tomcat 9 compliant configuration set.

Usability polish. A new Application Settings screen in the Options menu, a message variable legend on the monitor form, and changelog rendering fixes.

 

Reworked alert notification sequence

Previously, when a monitor tripped, QFlex persisted the alert to the database and then sent the notification. That ordering has a blind spot. If the host or database ran out of disk, the persist step failed and aborted the whole job, taking the alert email down with it. The exact moment you most needed to hear from QFlex was the moment it went quiet.

 

What changed in 4.6.3:

On the new alert path, the notification is now sent before the alert is persisted. The insert is wrapped so that a failed write is logged but no longer suppresses the alarm email. The notification does not depend on the generated alert ID, so the reordering is safe.

On the recovery path, the resolution update and the cleanup step are now wrapped the same way, and the recovery notification setting defaults to enabled if it cannot be read, so a write failure cannot swallow the all clear email either.

The result: transient or disk related database write failures degrade gracefully. You still get the alert, and you still get the all clear.

 

Monitoring watchdog (new)

The reordering above keeps alerts flowing when writes fail. But there is a harder case: the database becoming fully unreadable, where QFlex cannot even load the monitors it is supposed to run. In that state the normal alert engine has nothing to work with. The watchdog exists for exactly that dead zone.

The watchdog is a small, database-independent background job that runs on a fixed schedule, every 60 seconds by default. On each cycle it:

1. Probes the database with a cheap read to confirm QFlex can reach it.

2. Probes disk usage across all filesystem roots plus the temp and Tomcat directories, either auto detected or from a list you specify.

3. Decides and alerts. If the database has been unreachable for several consecutive checks, or a disk crosses the critical threshold, it emails an alert using SMTP settings it cached to a local file while healthy, so it can still send during a database outage, even across a restart.

Key design points:

Database independent by design. At notification time the watchdog never touches the database. It reads its SMTP settings from a local, owner readable file that it refreshes from QFlex on every healthy cycle.

Survives restarts during an outage. Because the SMTP settings are cached to disk, a watchdog restarted mid outage still knows how to reach you.

De-duplicated and self healing. While degraded it re-sends at most once every 15 minutes, configurable, and it sends a single monitoring recovered email once the database and disks are healthy again.

Early disk warning. Above the warn threshold, 85 percent by default, it logs loudly while everything still works, so you can act before it becomes critical.

Fails safe. The service is written never to throw. A probe failure is itself the signal, not a crash.

 

Platform refresh

Apache Tomcat 8.5.99 upgraded to 9.0.117, with a Tomcat 9 compliant configuration set.

MySQL Connector/J upgraded to 8.0.33 across all modules.

Default JVM heap set to 1 GB minimum and 2 GB maximum for predictable memory behavior out of the box.

 

Usability and administration

The Application Settings screen is now exposed as the first entry in the Options menu, available to administrators.

A message variable legend has been added to the monitor form, so the available substitution variables are documented right where you build alert messages.

Changelog rendering has been hardened against missing or empty queue manager data.

 

Upgrade notes

This release includes a platform upgrade to Tomcat 9. Review any custom Tomcat configuration against the refreshed 4.6.3 configuration set.

The watchdog is enabled by default and works with no configuration, using your existing QFlex SMTP settings. To tune thresholds, schedule, or recipients, copy watchdog.properties.sample to watchdog.properties and edit it. See the watchdog guide for details.

No database schema changes are required for the watchdog. It stores its cached state in a local properties file under the Tomcat base directory.