mask logs with logstash logback in java using regex

In today's data-driven world, data security is most important. Logging frameworks play a crucial role in application monitoring and debugging, but they can inadvertently expose sensitive information which should not have been. Log masking is a technique that effectively obfuscates sensitive data in log messages, safeguarding confidential information.

Understanding Logback

Logback is a powerful and mostly used logging framework in Java applications. It offers flexible configuration options, including the ability to format log events as JSON objects. It is a successor to the Log4j framework, has quickly gained popularity due to its features and ease of use. It is comprised of Logger, Encoders, Layout, Appender, Encoder.

Logger: Logger is a context for log messages. Applications will interact with this class to create log messages.

Encoders: Encoders are introduced in logback 0.9.91 and responsible for transforming an event into a byte array as well as writing out that byte array into an OutputStream. Encoders introduced as Layouts are only capable to transform an event into a String which restricts their scope to non-binary output.

Layout: The layout is responsible for formatting the logging request according to the user's wishes, whereas an appender takes care of sending the formatted output to its destination.

Appenders: In logback speak, an output destination is called an appender. This places log messages in their final destinations. A Logger can have more than one Appender. Currently, appenders exist for the console, files, remote socket servers, to MySQL, PostgreSQL, Oracle and other databases, JMS, and remote UNIX Syslog daemons.

About Logstash Logback Encoder

The logstash-logback-encoder library is a valuable tool for enhancing your Spring Boot applications' logging capabilities. It provides a convenient way to format log messages in a structured JSON format, making them readily consumable by log aggregation and analysis tools like Logstash. JSON format provides a structured and machine-readable way to log information, making it ideal for advanced log analysis and security measures. Benefits of logstash

• JSON Customization Logstash allows you to customize the JSON output to include specific fields and metadata.




• Dynamically Fields It also allows Dynamically add fields to log events based on application context.




• Improved Readability JSON format offers a clear and human-readable structure for log events.




• Enhanced Search and Analysis Log aggregation tools can easily parse and query JSON logs.




• Machine Parsing JSON logs are ideal for automated analysis and alerting systems.

Solution to mask the data in logs

Here main goal is to provide a solution to mask the data which is customizable and configurable at runtime.

Here is our simple requirement to

1. mask password completely in log.


2. mask phone number and login name except last 5 in log.

Step 1

Create spring boot application. This solution will work with any java based application with little customization.

Step 2

Configure all regular expressions to mask the data. Please remember regex are costly in terms of resource utilization. Make sure you are tuning your regex. Regex groups will allows us to select required substring from string.

This is very basics solution and having lot of scope for enhancement as per requirement like message digest etc...

You can find code at GitHub.

Please drop a comment if you have any question.