Integration of unified logging layer and centralized log management system into highly complex product tech stack

Integration of unified logging layer and Graylog/Logstash centralized log management system into highly complex product tech stack.

Some of the problems included:

Company’s product required unifying different logging mechanisms used and centralize output from these for online viewing
Company’s product highly complex tech stack, consisting of hundreds (300+) OSGi bundles, used five different logging frameworks – ‘logback’, ‘commons-logging’, ‘java.util.logging’, ‘tinylog’, and OSGi log service – which were configured separately and output diagnostic information to separate locations, causing significant loss of information and maintenance overhead
Due to incomplete diagnostic information it was much more difficult to trace root causes of some of the problems which occur
Diagnosing problems required direct access to Customer’s devices due to lack of centralized log management system where logs can be viewed online

Some of the solutions applied included:

Merging output from all of the logging frameworks used in Company’s product into one by introducing unified logging layer, utilizing “Pax Logging” 3rd party library
Sending merged output to Graylog/Logstash centralized log management system, utilizing Graylog/Logstash custom Logback appender, facilitating viewing of diagnostic information emitted by components of Company’s product in on-premise deployments via web browser
Removing all obsolete logging libraries and configuration files from bundles used by Company’s product
Updating build scripts, target definition and product definition files to reflect changes applied

Case Studies