In this post, I would like to show how I have implemented – getting real time results from JMeter using InfluxDB & Grafana.
This topic is huge – We can do a lot with this framework setup. I am unable to cover everything in this 1 article. so, I have tried to provide links to other articles of TestAutomationGuru which could be related to this article. Please check everything if possible.
When I used to run my JMeter test, I was unable to understand the performance of the application during the test execution. I always had to wait for the test to complete. It was little bit annoying – especially during long running tests. Sometimes, I might have to run the test for 2/3 days continuously.
At some point, JMeter had enabled the summariser output from its version 2.11 in Non-GUI mode. You could see below output when you run your JMeter test.
It was little bit helpful & above summary gives decent amount of information you would need while the test is running. It shows the minimum, maximum, average responses times, throughput, error count, no of active users for every 30 seconds (summary interval can be changed in jmeter.properties).
But when the test runs for hours & when you have huge amount of summariser output in the console for every 30 seconds, It might be a bit difficult to understand the results. For ex, If I would like to know the active user count at which the rate drops, I need to go through the summariser output line by line very carefully! I can not share the results with the development team/application architect as the results are not in a very user friendly format. To create a nice graph, I need to wait for the JMeter test to finish; So that I can access the jtl file & create graphs.
JMeter v2.13 has introduced a new listener to send the results to a time-series database (influxdb/graphite) while the test is running. By configuring Grafana (an open source metrics dashboard) to connect to the influxdb/graphite, We can create nice graphs while the JMeter is running the test!!
Time-Series Database: time series is the sequence of data taken over time. Time-series database is a software application which handles the time-series data. Imagine it like a sql-table in which time is the primary key!
We would be using InfluxDB as our database in this post. Check here for more information.
Few Terminologies you might want to know:
influxd -config /path/to/config/file
Once InfluxDB is up and running, then the next step is to create a JMeter test which sends the data to InfluxDB.
Note: Make sure that the system time of the Influx server is in sync with the machine which runs JMeter. This is very important. Because when you query Influxdb – by default it shows the records which has the time as <= system time. If the JMeter machine’s time is, say 4:50 PM. InfluxDB server system time is 4:45PM. When you run JMeter test, It will be able to post the records to InfluxDB. However when you query influxdb, It will not show these records. You need to wait for 5 mins to see the records inserted.
Even after following all the steps, It does not work as expected? No problem; Check here for the steps to troubleshoot.
Hope you are able to send the test results to InfluxDB successfully!!! Then we need to setup Grafana server to create charts from the test results.
Hopefully you got the idea now!! You can go from here to create different metrics as you want!!
Eventhough the new backend listener is good, I feel it has few drawbacks. It does not give me the stats I was looking for. I was actually looking for
So, in order to meet my goals, I just wanted to go with my own implementation – by updating the Apache Core library which has the summariser (it is responsible the writing the above info in the console) – So whenever it writes the info in the console and it can also update the data in the Influxdb. Not creating a separate listener for this purpose seems to be ok to me as adding more listeners to the test will consume memory and might affect my test. Performance wise – this approach performs better than the adding new backend listener because I do not have to add any additional logic for the calculation.
Advantages of modifying summariser to send the results to InfluxDB:
From below comments, many of you seem to be interested in trying the above approach of mine. So, I would like to share the Apache Core library (jar) + Grafana dashboard exported in json format.
#True to send data to influx db summariser.influx.out.enabled=true # influxdb server ip summariser.influx.ip=10.11.12.13 # influxdb port to post the data via http summariser.influx.port=8086 # influxdb database summariser.influx.db=jmeter # name of the project. might be useful when you use same DB for multiple projects. # it is stored as tag for faster query summariser.influx.project=taguru # name of the test suite. might be useful when you run the same test for different conditions. # it is stored as tag for faster query summariser.influx.project.suite=taguru-100-users # timeouts summariser.influx.connection.timeout=5000 summariser.influx.socket.timeout=5000
When you use above library and the properties, you might see below 3 measurements created in the InfluxDB.
If you are still facing issues in sending the data to InfluxDB - Check this post to troubleshoot.
The downloaded zip file should also contain the Grafana dashboards (in JSON format). Import those dashboard files in Grafana (You can find the Import feature under Home as shown below).
Once it successfully imports, It will create dashboards as shown below. It takes the data from delta and total measurements to show the appropriate metrics. (Make sure that your Grafana has the datasource set correctly. It should point to the influxDB instance where we post the data).
This dashboard will retrieve the data from Samples measurement. You can filter further based on specific page, their status etc. Sometimes, when the request fails (say some auth error), the response time might be very less – so throughput will be huge. This is not right. This is why I have the status filter to get the success only throughput.
By adjusting the ‘Summarize’ drop down, Let grafana calculate the throughput for different intervals.
Using JMeter’s Backend listener / Custom listener + InfluxDB + Grafana – we can create stunning charts which will give us the real time metrics of the test which is being run by JMeter. If you have a long running tests, You do not need to wait for the test to finish to access the results. Just share the Grafana dashboard URL – so that anyone with link can see what is going on!
Please check below articles as well which are related to this.
Happy Testing & Subscribe 🙂