Karate Series | Session 7 | Karate Reports

What we explore in this series

Karate Basics
GET API
POST API
PUT API
DELETE API
Karate Config
Karate Reports

Karate has a built-in native HTML reporting feature, which should ideally serve all your needs. Still, you can enable cucumber reporting, which would also generate reports in the form of HTML.

If you wanted to enable the cucumber reporting feature, the following dependency to be added to the test scope. We had already discussed this in brief in the first session of this series.

<dependency>
    <groupId>net.masterthought</groupId>
    <artifactId>cucumber-reporting</artifactId>
    <version>5.3.0</version>
    <scope>test</scope>
</dependency>

We can easily integrate karate reporting with the CI tools as most of such tools would be able to process the output of the parallel runner and determine the status of the build as well as generate reports.

The native reporting feature does not only create an HTML file but also creates feature-wise output files in the form of XML and JSON. These output files will be consumed by third-party plugins like cucumber and generate catching HTML reports of their own.

There also exists another maven reporting plugin that is compatible with Karate JSON called Cluecumber.

Karate Reports

Once you run the TestRunner class (we’ve already gone through TestRunner in the first session of this series), all the feature files will get executed, and an HTML file called karate-summary.html will be generated in the following path.

<project-root-directory>/target/surefire-reports

Apart from this summary file, feature-wise JSON and XML files will also be generated in the same directory. The directory will also contain another HTML file called timeline.html which shows an overall picture of the parallel tests done and says how much time was taken for each test.

Here’s the karate-summary.html

This is what we get if we click on one of the features given in the above picture. All the steps for each scenario belonging to that feature will be listed and the time taken for each step will also be given.

Here’s the timeline.html

Cucumber Reports

If the cucumber reporting feature is enabled and the TestRunner gets run, it will create a file named overview-features.html in the following path. The path is customizable, and we can set this output directory location in the Configuration class of the Cucumber plugin.

<project-root-directory>/src/test/java/reports/cucumber-html-reports

Cucumber will read the JSON files generated by the Karate and generate a report of its own in this directory. So the code for generating cucumber reports has been written in the TestRunner class and the class would look like this.

import com.intuit.karate.KarateOptions;
import com.intuit.karate.Results;
import com.intuit.karate.Runner;
import com.intuit.karate.junit4.Karate;
import net.masterthought.cucumber.Configuration;
import net.masterthought.cucumber.ReportBuilder;
import org.apache.commons.io.FileUtils;
import org.junit.BeforeClass;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;

import java.io.File;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;


@RunWith(Karate.class)
@KarateOptions(features = "classpath:features", tags = "~@ignore")
public class TestRunner {

    @BeforeClass
    public static void init() {
        System.setProperty("karate.env", "local");
    }

    @Test
    public void testParallel() {
        Results results = Runner.path("classpath:features").tags("~@ignore").parallel(4);
        generateReport(results.getReportDir());
    }

    public static void generateReport(String outputPath) {
        Collection<File> jsonFiles = FileUtils.listFiles(new File(outputPath), new String[]{"json"}, true);
        List<String> jsonPaths = new ArrayList<>(jsonFiles.size());
        jsonFiles.forEach(file -> jsonPaths.add(file.getAbsolutePath()));
        Configuration config = new Configuration(new File("./src/test/java/reports"), "karate-api-automation");
        new ReportBuilder(jsonPaths, config).generateReports();
    }
}

This is how cucumber report looks like

If we click on one of the features, like Karate reports, each individual step can be seen.

If one of the tests gets failed, it will report like this

Finally, this is the project structure

So, here the Karate series has come to an end. We will catch up with another one soon.