Zalando’s Logbook: A Developer’s Guide with Code Examples

HTTP request and response logging

Logging is a fundamental aspect of software development. It provides developers with insights into how their applications are behaving, helps debug issues, and monitors a system's health. Zalando, one of Europe’s largest online fashion retailers, has developed an open-source logging library called “Logbook.”

In this article, we’ll dive into Zalando’s Logbook, explore its features, and provide code examples to help developers implement effective logging in their applications.

What is logbook?

Logbook is an extensible Java library to enable complete request and response logging for different client- and server-side technologies. It satisfies a special need by a) allowing web application developers to log any HTTP traffic that an application receives or sends b) in a way that makes it easy to persist and analyze it later. This can be useful for traditional log analysis, meeting audit requirements or investigating individual historic traffic issues. — source

Here are some of its key features:

1. Structured Logging: Logbook encourages structured logging by allowing developers to attach key-value pairs (contextual information) to log messages. This makes it easier to search, filter, and analyze log data.
2. Request and Response Logging: Logbook is tailored for web applications, making it straightforward to log HTTP request and response details, including headers and body content.
3. Streaming Support: Logbook supports various output streams like files, standard output (stdout), and standard error (stderr). You can configure multiple handlers to send logs to different destinations.
4. Extensibility: Developers can extend Logbook’s functionality by creating custom log handlers and formatters, allowing you to integrate with various logging services and formats.

Basic setup in spring boot app

Dependency to include in your pom.xml or gradle.build file.

<dependency>
    <groupId>org.zalando</groupId>
    <artifactId>logbook-spring-boot-starter</artifactId>
    <version>${logbook.version}</version>
</dependency>

implementation group: 'org.zalando', name: 'logbook-spring-boot-starter', version: '' // Replace with the latest version

Logging level to set in application.yml

logging:
  level:
    org.zalando.logbook: TRACE

Result in logs after executing an HTTP request

2023-09-12T13:26:40.853+02:00 TRACE 66142 --- [nio-8080-exec-1] org.zalando.logbook.Logbook              : {"origin":"remote","type":"request","correlation":"e5d4014e426ef77c","protocol":"HTTP/1.1","remote":"0:0:0:0:0:0:0:1","method":"POST","uri":"http://localhost:8080/api/v1/account/create-mock-accounts","host":"localhost","path":"/api/v1/account/create-mock-accounts","scheme":"http","port":"8080","headers":{"accept":["*/*"],"accept-encoding":["gzip, deflate, br"],"cache-control":["no-cache"],"connection":["keep-alive"],"content-length":["0"],"host":["localhost:8080"],"postman-token":["c4426513-9c1d-48a8-8e97-baf24fd8e2a8"],"user-agent":["PostmanRuntime/7.32.3"]}}

Elasticsearch Integration

To store logs in Elasticsearch, you can use the Elasticsearch Logbook Logstash dependency. Add the following dependency to your Maven or Gradle configuration:

<dependency>
    <groupId>org.zalando</groupId>
    <artifactId>logbook-logstash</artifactId>
    <version>${logbook.version}</version>
</dependency>

After adding the dependency, configure Elasticsearch in your logbook.properties file:

# logbook.properties
logbook.logstash.target=elasticsearch
logbook.logstash.hosts=http://localhost:9200
logbook.logstash.index=logbook

Viewing Logs in Elasticsearch

Once your application is up and running, it will log requests and responses to Elasticsearch. You can use Kibana or any other Elasticsearch log viewer to analyze and search through your logs effectively.

Customization of Logbook

Filtering

There are default values filtered from logbook’s logging, but you can customize it further. Filtering aims to avoid sensitive data logging since no JWT token or client secret should be there in the logs. Additionally, since GDPR came into the picture, you need to know which data stored where. If you are logging data without any sense you won’t be able to comply with the GDPR rules with deleting certain data.

Logbook supports different types of filters like QueryFilter, PathFilter, HeaderFilter, BodyFilter, RequestFilter and ResponseFilter .

Probably you need to customize the latter two meaning Request- and ResponseFilter.

Example for customization

Logbook logbook = Logbook.builder()
        .requestFilter(RequestFilters.replaceBody(message -> contentType("audio/*").test(message) ? "mmh mmh mmh mmh" : null))
        .responseFilter(ResponseFilters.replaceBody(message -> contentType("*/*-stream").test(message) ? "It just keeps going and going..." : null))
        .queryFilter(accessToken())
        .queryFilter(replaceQuery("password", "<secret>"))
        .headerFilter(authorization())
        .headerFilter(eachHeader("X-Secret"::equalsIgnoreCase, "<secret>"))
        .build();

Furthermore, you can use jsonPath as a Java DSL for JSON.

Logbook logbook = Logbook.builder()
        .bodyFilter(jsonPath("$.password").delete())
        .bodyFilter(jsonPath("$.active").replace("unknown"))
        .bodyFilter(jsonPath("$.address").replace("X"))
        .bodyFilter(jsonPath("$.name").replace(compile("^(\\w).+"), "$1."))
        .bodyFilter(jsonPath("$.friends.*.name").replace(compile("^(\\w).+"), "$1."))
        .bodyFilter(jsonPath("$.grades.*").replace(1.0))
        .build();

Formatting

You can format the logging, the default is HTTP, but using

• JsonHttpLogFormatter,
• CommonsLogFormatSink
• ExtendedLogFormatSink
• CurlHttpLogFormatter
• SplunkHttpLogFormatter (key-value format)

Wrap it up

Zalando’s Logbook is a powerful logging library for developers, especially those working on web applications. With its structured logging capabilities, request and response logging, and extensibility, Logbook makes it easier to maintain and analyze logs, leading to better application performance and debugging experiences. By integrating it with Elasticsearch, you can centralize log storage and easily analyze logs for debugging and monitoring purposes.

Whether you need to log HTTP requests, design custom log handlers, or simply want to enhance your logging capabilities, Logbook can be a valuable addition to your toolkit. Start using Logbook today to streamline your logging process and gain better insights into your application’s behavior.

Please clap, follow me and share this article if you liked it. ❤

Have a nice day! Let’s code!

Sources

https://github.com/zalando/logbook