avatarEthiraj Srinivasan

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

2270

Abstract

on.</p><h2 id="a53a">Installation:</h2><p id="0ff0">Memory Profiler can be installed from PyPl using:</p><div id="de37"><pre>pip <span class="hljs-keyword">install</span> -U memory_profiler</pre></div><p id="d187">and can be imported using</p><div id="3752"><pre><span class="hljs-keyword">from</span> memory_profiler <span class="hljs-keyword">import</span> profile</pre></div><h2 id="6156">Usage:</h2><p id="2314">After everything is set up, it's pretty easy to use this module to track the memory consumption of the function. <code>@profile</code> decorator can be used before every function that needs to be tracked. This will track the memory consumption line-by-line in the same way as of <a href="https://pypi.org/project/line-profiler/">line-profiler</a>.</p><p id="f468">After decorating all the functions with <code>@profile</code> execute the python script with a specific set of arguments.</p> <figure id="b96f"> <div> <div>

            <iframe class="gist-iframe" src="/gist/satkr7/0102b4e5e2acff9db15809298e77a1d1.js" allowfullscreen="" frameborder="0" height="undefined" width="undefined">
          </div>
        </div>
    </figure></iframe></div></div></figure><p id="7b9f">Execute the above Python script using bypassing <code>-m memory profiler</code> to the Python interpreter. This will load the memory_profiler module and print the memory consumption line-by-line.</p><p id="ebf8">Use the below to execute the Python script along with the memory profiler.</p><div id="f938"><pre><span class="hljs-keyword">python</span> -<span class="hljs-keyword">m</span> memory_profiler <span class="hljs-symbol">&lt;filename&gt;</span>.<span class="hljs-keyword">py</span></pre></div><figure id="8ba9"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*--WZVx_xXuDeyE2slwvmjA.png"><figcaption>(Image by Author)</figcaption></figure><p id="6a84">After successful execution, you will get a line-by-line memory consumption report, similar to the above image. The report has 5 columns:</p><ul><li><b>Line #</b>: Line Number</li><li><b>Line Contents</b>: Python code at each line number</li><li><b>Mem usage</b>: Memory usage by the Python interpreter after every execution of the li

Options

ne.</li><li><b>Increment</b>: Difference in memory consumption from the current line to the last line. It basically denotes the memory consumed by a particular line of Python code.</li><li><b>Occurrences</b>: Number of times a particular line of code is executed.</li></ul><p id="fd15">Mem Usage can be tracked to observe the total memory occupancy by the Python interpreter, whereas the Increment column can be observed to see the memory consumption for a particular line of code. By observing the memory usage one can optimize the memory consumption to develop a production-ready code.</p><h1 id="f44f">Conclusion:</h1><p id="dbbd">Optimizing the memory consumption is as important as optimizing the time complexity of the Python code. By optimizing the memory consumption, one can speed up the execution to some extent and avoid memory crashes.</p><p id="9278">One can also try custom <code>@profile</code> decorators to specify the precision of the argument. Read the <a href="https://pypi.org/project/memory-profiler/">documentation</a> of the memory profiler module for better understanding.</p><h1 id="3c87">References:</h1><p id="e914">[1] Memory Profiler Documentation: <a href="https://pypi.org/project/memory-profiler/">https://pypi.org/project/memory-profiler/</a></p><p id="ff83"><i>Loved the article? Become a <a href="https://satyam-kumar.medium.com/membership">Medium member</a> to continue learning without limits. I’ll receive a small portion of your membership fee if you use the following link, with no extra cost to you.</i></p><div id="fbae" class="link-block"> <a href="https://satyam-kumar.medium.com/membership"> <div> <div> <h2>Join Medium with my referral link - Satyam Kumar</h2> <div><h3>As a Medium member, a portion of your membership fee goes to writers you read, and you get full access to every story…</h3></div> <div><p>satyam-kumar.medium.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*sp1Stkiu2tDeRpx8)"></div> </div> </div> </a> </div><p id="e94c" type="7">Thank You for Reading</p></article></body>

Custom actuator metrics for better spring boot application monitoring

Use these 3 steps to add custom actuator Prometheus metrics for better spring boot application monitoring

Photo by Michał Jakubowski on Unsplash

This article describes an elegant way to add a custom actuator Prometheus metric to spring boot applications. Spring boot, a powerful java based framework to develop microservices web applications provides many easy-to-configure solutions for developer's needs.

Developing a microservices application is complete only when we can monitor these microservices. Spring boot provides Actuator to help monitor these microservices. Actuator uses HTTP endpoints to provide information about the application. The actuator provides many metrics like database connections, thread information, memory information, HTTP client and server requests, etc. Below are the 3 steps to add custom actuator Prometheus metrics in the spring boot

  1. Enable actuator in spring boot application
  2. Add Prometheus micrometer to the application
  3. Add custom metrics to micrometer Prometheus

1. Enable Actuator in Spring Boot Application

To add an actuator to the spring boot application add the following to pom.xml

<dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Run your spring application using

mvn spring-boot:run

The application will run on port 8080 by default. Go to http://localhost:8080/actuator to view the actuator endpoints like below.

Actuator End Points

The actuator provides a health check endpoint that shows the status of the application. The status is “UP” if the application is healthy and “DOWN” if there are any issues

Actuator Health End Point

The actuator does provide more than just health information. To unlock its true potential expose its endpoint to the outside world. To do this, include the following in application.properties

management.endpoint.info.enabled=true 
management.endpoints.web.exposure.include=*

Now go to http://localhost:8080/actuator to see more actuator endpoints exposed like below

{"_links":{"self":{"href":"http://localhost:8080/actuator","templated":false},
"beans":{"href":"http://localhost:8080/actuator/beans","templated":false},
"caches-cache":{"href":"http://localhost:8080/actuator/caches/{cache}","templated":true},
"caches":{"href":"http://localhost:8080/actuator/caches","templated":false},
"health-path":{"href":"http://localhost:8080/actuator/health/{*path}","templated":true},
"health":{"href":"http://localhost:8080/actuator/health","templated":false},
"info":{"href":"http://localhost:8080/actuator/info","templated":false},
"conditions":{"href":"http://localhost:8080/actuator/conditions","templated":false},
"configprops":{"href":"http://localhost:8080/actuator/configprops","templated":false},
"configprops-prefix":{"href":"http://localhost:8080/actuator/configprops/{prefix}","templated":true},
"env":{"href":"http://localhost:8080/actuator/env","templated":false},
"env-toMatch":{"href":"http://localhost:8080/actuator/env/{toMatch}","templated":true},
"loggers":{"href":"http://localhost:8080/actuator/loggers","templated":false},
"loggers-name":{"href":"http://localhost:8080/actuator/loggers/{name}","templated":true},
"heapdump":{"href":"http://localhost:8080/actuator/heapdump","templated":false},
"threaddump":{"href":"http://localhost:8080/actuator/threaddump","templated":false},
"metrics":{"href":"http://localhost:8080/actuator/metrics","templated":false},
"metrics-requiredMetricName":{"href":"http://localhost:8080/actuator/metrics/{requiredMetricName}","templated":true},
"scheduledtasks":{"href":"http://localhost:8080/actuator/scheduledtasks","templated":false},
"mappings":{"href":"http://localhost:8080/actuator/mappings","templated":false}}}

As seen above the actuator provides information about beans, env, heap dump, thread dump, metrics, etc. To get any detailed information click on the http link. In the case of a thread dump click: http://localhost:8080/actuator/threaddump

The actuator shows thread dump information

2. Add Micrometer Prometheus-based metric to the application

Next to publish the metrics to monitoring tools like Prometheus, Datadog, or Dynatrace a standard format is required. Micrometer acts as that intermediate layer to easily interface with the monitoring tools. Prometheus is chosen for this example which performs time series analysis and uses a pull-based approach. Simply add a micrometer Prometheus setup to the application by including the below dependency

<dependency>
 <groupId>io.micrometer</groupId>
 <artifactId>micrometer-registry-prometheus</artifactId>
 <scope>runtime</scope>
</dependency>

Now rerun the application to see the new actuator point — http://localhost:8080/actuator/prometheus.

Actuator Prometheus endpoint information

3. Add custom metrics to micrometer Prometheus

Now that the default Prometheus metrics are available time to add the custom metrics. Custom metrics can be added in service or business logic classes. Two objects are essential to building the custom metric

  1. MeterRegistry
  2. Metric Type Object — Counter/Gauge/Timer
  • Use Spring boot @Autowired annotation to create MeterRegistry
    @Autowired
    MeterRegistry registry;
  • Next, initialize the metric type in the constructor or post-construct methods like below
@PostConstruct
public void initialize(){
 counter = Counter.builder("custom_counter_metric").register(registry);
 Gauge.builder("custom_gauge_metric",numbersList,Collection::size).register(registry);
 timer = Timer.builder("custom_timer_metric").register(registry);
}
  • Record the metrics in methods as follows
# Counter
//custom logic
counter.increment();
counter.increment(5);
# Timer
//custom logic
timer.record(Duration.between(tenMinuteBefore,LocalDateTime.now()).getSeconds(), TimeUnit.SECONDS);

A sample service class is as follows

@Service
public class DemoService {

    @Autowired
    MeterRegistry registry;

    private Counter counter;

    private ArrayList numbersList = new ArrayList<>();

    private Timer timer;


    @PostConstruct
    public void initialize(){
       counter = Counter.builder("custom_counter_metric").register(registry);
       Gauge.builder("custom_gauge_metric",numbersList,Collection::size).register(registry);
       timer = Timer.builder("custom_timer_metric").register(registry);
     }

    public void incrementCounter() {
        //custom logic
        counter.increment();
        counter.increment(5);
    }

    public void processGauge() {
        numbersList.add(new Random());
    }

    public void recordTime() {
        LocalDateTime tenMinuteBefore = LocalDateTime.now().minusMinutes(10);
        //custom logic
        timer.record(Duration.between(tenMinuteBefore,LocalDateTime.now()).getSeconds(), TimeUnit.SECONDS);
    }
}

Once these methods are called navigate to http://localhost:8080/actuator/prometheus/ to see the custom metrics

# HELP custom_counter_metric_total 
# TYPE custom_counter_metric_total counter 
custom_counter_metric_total 6.0
# HELP custom_gauge_metric  
# TYPE custom_gauge_metric gauge
custom_gauge_metric 1.0
# HELP custom_timer_metric_seconds_max  
# TYPE custom_timer_metric_seconds_max gauge
custom_timer_metric_seconds_max 0.0
# HELP custom_timer_metric_seconds  
# TYPE custom_timer_metric_seconds summary
custom_timer_metric_seconds_count 1.0
custom_timer_metric_seconds_sum 600.0

Thus custom metrics are added to the spring boot application. Happy Monitoring !!!

Originally published at https://ethigeek.com.

Thank you for your comments and for sharing my stories to reach a broader audience.

If you like my article and would like to support me, make sure to:

Java
Spring
Monitoring
Development
Microservices
Recommended from ReadMedium