JUnit's stopwatch to capture time taken per integration test in csv file

I was looking to improve my integration test suite, which contains large number of integration test (more than 900 test cases). My immediate focus was to tackle the slowest running tests in my test suite.

TestNG reports provide the information of slowest running test out of the box, however my project was using JUnit 4. Being a lazy developer, I didn't want to migrate my entire test suite to TestNG in order to get reports about slowest running test. Hence I was looking at other options available in JUnit test framework.

JUnit offers a Rule called stopwatch, which notifies us the time taken to complete each test as callback methods. However, most of the examples showed how to print the information on console or some logger. Printing on the console or logger is particularly not very helpful, as it doesn't allow us to analyse the data in an easy way.

Hence I had created a custom stopwatch rule which extends JUnit's stopwatch rule and capture the information provided by JUnit as a csv file, which can then be used for further analysis (e.g sort tests by descending order of time elapsed) through spreadsheet softwares like Microsoft's Excel or Libre Office's Calc or Apple's Numbers.

The csv file provides two vital information:

• Chronological order of executed tests
• Total time taken (in millisecond) per test

Here is the sample code of CSV writer created with Java 8 and Junit 4.

TestStopwatchCsvWriter.java

package com.example.integration.test.util;

import org.junit.rules.Stopwatch;
import org.junit.runner.Description;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;

public class TestStopwatchFileWriter extends Stopwatch {
    private static final String MESSAGE_FORMAT = "%s%s%s\n";
    private final String reportFile;
    private final String valueSeparator;

    public TestStopwatchFileWriter(String reportLocation, String reportNameWithExtension, String valueSeparator) {
        this.reportFile = reportLocation + "/" + reportNameWithExtension;
        this.valueSeparator = valueSeparator;
        try {
            createReportFileIfNotPresent(reportLocation, reportFile);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    protected void finished(long nanos, Description description) {
        try {
            writeToReport(description.getDisplayName(), nanos);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    private void writeToReport(String testName, long timeTakenInNanos) throws IOException {
        Path path = Paths.get(reportFile);
        long timeTakenInMillis = timeTakenInNanos / 1000000;
        byte[] strToBytes = String.format(MESSAGE_FORMAT, testName, valueSeparator, timeTakenInMillis).getBytes(StandardCharsets.UTF_8);
        Files.write(path, strToBytes, StandardOpenOption.WRITE, StandardOpenOption.APPEND);
    }

    private void createReportFileIfNotPresent(String reportDirectory, String reportFile) throws IOException {
        if (!Files.exists(Paths.get(reportDirectory))) {
            Files.createDirectories(Paths.get(reportDirectory));
        }
        if (!Files.exists(Paths.get(reportFile))) {
            Files.createFile(Paths.get(reportFile));
        }
    }
}

Usage of csv writer in integration test:

Most projects will have a super base integration test class to do common setup and configuration, it will be the best place to apply this rule in this common super class, so that this csv rule will be applied centrally to all tests.

package com.example.integration.test;
import com.example.integration.test.util.TestStopwatchFileWriter;

public abstract class AbstractBaseIntegrationTest {
    // some existing code
    
    // For maven projects
    private String reportLocation = "./target/reports";
    // For gradle projects
    // private String reportLocation = "./build/reports";

    @Rule
    public TestStopwatchFileWriter testCsvWriter = new TestStopwatchFileWriter(reportLocation, "time_taken_per_integration_test.csv", ",");

    // some existing code
}

I hope this article might help someone while debugging integration tests during the improvement process.