We programmers never make mistakes. Or so we think and say. However, from time to time errors manage to sneak to production systems and no one seems to know how. Well MoSKito won’t change this, but it can help you hunting errors. And here is how.

There are several cool new features in the upcoming MoSKito release – 2.8.0. Errorcatching is probably the most helpful in the day-to-day work. First of all there is a new BuiltInProducer, the ErrorProducer.

The ErrorProducer

The ErrorProducer is always there and intercepts all exceptions and other throwables in moskito-monitored components. The ErrorProducer can be found in the producers overview:

The ErrorProducer distinguishes between two types of errors:

• Initial errors are first errors to happen in a processing thread
• Total errors are all errors in a thread, including first and followup errors.

For both values we also have the max-reached values. This is helpful to know if the currently reached value is reason to be worried or not. The max reached values are also used to create two standard gauges First Errors and Total Errors:

The gauges are created automatically and can be added to a Dashboard for better system overview. What is also created automatically are charts for the cumulated errors, for both initial and total values:

This nice chart will also make a great addition to a system dashboard.

But there is more.

Exceptions and Throwables.

Not only MoSKito is able to count the total amount of exceptions,MoSKito is also capable of doing it per-error. Same statistics as for cumulated errors are available for every exception caught in the system. Just click on the ErrorProducer to see the details.

The example only contains a couple of exceptions since its taken on a simple demo system, but in real production system every error will be shown. In the default interval setting all errors in the system are visible, for initial and total values usual interval selectors apply. The above gauges use 1m value for presentation.

Since every exception is also a statistic value in the ErrorProducer you can add accumulator to every exception you’d like to chart. Or, if you want to chart them all, simply set the configuration value autoChartErrors to true. In this case charts will be created automatically for every exception type:

Exception hunting

Knowing what kind of exception happen in my system is a good thing, but it still doesn’t explain where or why. However, we prepared some tools to help you find out:

Logging

If the configuration option logErrors is enabled, all errors MoSKito ‘sees’ will be logged into a logfile named auto-caught-errors.log. The name of the file can be adjusted in the logging configuration, for example logback.xml if you use logback.

Catching

Logging out all exceptions can be helpful, but not if there are too many. If you have too many exceptions or do not want to look into log files, you will love Catchers. 

A catcher is like a breakpoint for exceptions. Except it doesn’t stop the execution, but memorizes the exception. The catcher can log the specific exception into a separate logfile or keep it in memory or both. For details on configuration please visit the MoSKito Configuration Guide, but a quick example follows:

"@catchers": [
    {
      "clazz": "java.lang.IllegalArgumentException",
      "target": "LOGANDMEMORY",
      "parameter": "IllegalArgExcLog"
    }
}

The above fragment will force MoSKito both log and keep in memory all errors. The total amount of errors kept in memory is configurable, so don’t worry about your memory.

Not only the stack traces themselves are kept, but also all assigned tags, like customer number or email-address. A runtime catcher looks like this:

Obviously clicking on a specific catcher leads to the list of caught exceptions:

and click on the exception line opens the stacktrace:

Now, the unwanted exceptions have no place to hide anymore.

This is a brief overview of the new features for error monitoring in upcoming 2.8.0 release. No, we probably won’t prevent exceptions and bugs from sneaking into production, but finding them will be a lot easier!