I'm trying to come up with an alert that will notify me when the agent stops responding to SolarWinds (or stops sending data to SolarWinds).
First a bit of background:
I have a customer with 500+ agents and they have had numerous issues with the agents stop responding. Fixes have included reinstalling the agents, restarting the agents, or just removing the agent completely and reverting to WMI polling. The agent management service seems to randomly crash (support case open) and all of the agents report as down, flooding the system with node down alerts. The current workaround for this is to use ICMP polling for status but the new problem is there is no notification if an agent stops responding. The connection status says connected but clicking on a node will show response time/packet loss and intermittently no CPU, memory, volume, or application data. This results in missing data in charts, applications unknown, etc.
Has anyone created any alerts that trigger when an agent has not returned data within a particular time frame, e.g. 15 minutes?