Blazemeter Taurus load testing and offline jmeter html report
Recently I had used Blazemeter Taurus for load testing my application and I was quiet impressed with the features offered by Taurus. It is not a new tool by itself, instead it makes use of existing load testing tools like Apache JMeter or Gatling. However it offers enough abstraction to users from the underlying load testing tool using YAML configuration files.
2) Easy to review changes made to the Taurus YAML file on commit by commit basis to understand the changes made to the load testing. This is especially handy during pull request process
3) Easy to run in non-interactive command line mode, so easy to integrate with CI tools
4) Official docker image provided by Blazemeter Taurus, which required minimal setup to run load test in development machines and also in CI environment or cloud providers like AWS or AZURE which provisions temprovary VMs based on docker commands to simulate distributed load testing
Adding the following shellexec module in Taurus YAML help to auto generate html report after the test completion.
services:
- module: shellexec
post-process:
- awk -F',' -v OFS=',' '$7 = $7 FS "failureMessage"' ${TAURUS_ARTIFACTS_DIR}/kpi.jtl > ${TAURUS_ARTIFACTS_DIR}/jmeter_stats.jtl
- find ~/.bzt -type f -name "jmeter" -exec ln -s {} . ';'
- ./jmeter -g ${TAURUS_ARTIFACTS_DIR}/jmeter_stats.jtl -o ${TAURUS_ARTIFACTS_DIR}/jmeterHtmlReports
Since the above command uses unix commands like awk and find, it is easier the run the Taurus load test using the official Taurus docker, which is based on unix OS.
Running in docker removes the hassle of setting up blazemeter and enables to quicly setup distributed or clustered load testing in cloud providers like AWS or Azure.
A sample github repository to demonstration the auto generate of html report through docker is available in the following url.
Taurus Load testing demo: https://github.com/harishkannarao/TaurusLoadTesting
The repository also tells how to generate unified html report for clustered or distributed load testing
Advantages of using Taurus over JMeter
1) JMeter uses single jmx xml file to simulate the load test, which makes it hard for more than one person to work on the single jmx file and resolving conflicts might turn tricky2) Easy to review changes made to the Taurus YAML file on commit by commit basis to understand the changes made to the load testing. This is especially handy during pull request process
3) Easy to run in non-interactive command line mode, so easy to integrate with CI tools
4) Official docker image provided by Blazemeter Taurus, which required minimal setup to run load test in development machines and also in CI environment or cloud providers like AWS or AZURE which provisions temprovary VMs based on docker commands to simulate distributed load testing
Missing feature in Taurus
Given said about the advantages of Taurus, not everything is perfect. One key missing feature in Taurus over Gatling is the auto html report generation. By default, Taurus generates the statistics file kpi.jtl but doesn't generate html report out of it. However it is not end of the world. Blazemeter provides their online service to generate html report from the kpi.jtl. If you project / application is sensitive, then you can generate offline html reports using JMeter from the kpi.jtlAdding the following shellexec module in Taurus YAML help to auto generate html report after the test completion.
services:
- module: shellexec
post-process:
- awk -F',' -v OFS=',' '$7 = $7 FS "failureMessage"' ${TAURUS_ARTIFACTS_DIR}/kpi.jtl > ${TAURUS_ARTIFACTS_DIR}/jmeter_stats.jtl
- find ~/.bzt -type f -name "jmeter" -exec ln -s {} . ';'
- ./jmeter -g ${TAURUS_ARTIFACTS_DIR}/jmeter_stats.jtl -o ${TAURUS_ARTIFACTS_DIR}/jmeterHtmlReports
Since the above command uses unix commands like awk and find, it is easier the run the Taurus load test using the official Taurus docker, which is based on unix OS.
Running in docker removes the hassle of setting up blazemeter and enables to quicly setup distributed or clustered load testing in cloud providers like AWS or Azure.
A sample github repository to demonstration the auto generate of html report through docker is available in the following url.
Taurus Load testing demo: https://github.com/harishkannarao/TaurusLoadTesting
The repository also tells how to generate unified html report for clustered or distributed load testing
HI Harish,
ReplyDeleteI have run docker-compose up and I see this error, Not able to understand, please help
> docker-compose up
Creating blazemeter-taurus-load-test ... error
ERROR: for blazemeter-taurus-load-test Cannot create container for service blazemeter-taurus-load-test: status code not OK but 500: ☺ ˙˙˙˙☺ ♀☻ FDocker.Core, Version=3.0.2.51106, Culture=neutral, PublicKeyToken=null♣☺ ←Docker.Core.DockerException♀ ClassNameMessage♦Data♫InnerExceptionHelpURL►StackTraceString▬RemoteStackTraceString►RemoteStackIndex☼ExceptionWatsonBuckets☺☺♥♥☺☺☺ ☺ ☺▲System.Collections.IDictionary►System.Excepti☻☻ ♠♥ ←Docker.Core.DockerException♠♦ ▲Filesharing has been cancelled
♠♣ Ś‼ at Docker.ApiServices.Mounting.FileSharing.d__8.MoveNext() in C:\workspaces\PR-15138\src\github.com\docker\pinata\win\src\Docker.ApiServices\Mounting\FileSharing.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Docker.ApiServices.Mounting.FileSharing.d__6.MoveNext() in C:\workspaces\PR-15138\src\github.com\docker\pinata\win\src\Docker.ApiServices\Mounting\FileSharing.cs:line 55
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Docker.HttpApi.Controllers.FilesharingController.d__2.MoveNext() in C:\workspaces\PR-15138\src\github.com\docker\pinata\win\src\Docker.HttpApi\Controllers\FilesharingController.cs:line 21
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Threading.Tasks.TaskHelpersExtensions.d__1`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ApiControllerActionInvoker.d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ActionFilterResult.d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Dispatcher.HttpControllerDispatcher.d__15.MoveNext()
♠♠ Ł☺8
MoveNext
Docker.ApiServices, Version=3.0.2.51106, Culture=neutral, PublicKeyToken=null
Docker.ApiServices.Mounting.FileSharing+d__8
Void MoveNext() §‼€♠ ↕Docker.ApiServices
ERROR: for blazemeter-taurus-load-test Cannot create container for service blazemeter-taurus-load-test: status code not OK but 500: ☺ ˙˙˙˙☺ ♀☻ FDocker.Core, Version=3.0.2.51106, Culture=neutral, PublicKeyToken=null♣☺ ←Docker.Core.DockerException♀ ClassNameMessage♦Data♫InnerExceptionHelpURL►StackTraceString▬RemoteStackTraceString►RemoteStackIndex☼ExceptionWatsonBuckets☺☺♥♥☺☺☺ ☺ ☺▲System.Collections.IDictionary►System.Excepti☻☻ ♠♥ ←Docker.Core.DockerException♠♦ ▲Filesharing has been cancelled
Sorry , my bad. The issue was with Docker WSL settings. Resolved now.
ReplyDeleteThanks
Sri