A client of mine wanted to measure coverage of certain modules. The best tool to measure coverage of JVM code is JaCoCo which is easy to use from any kind of build tool, be it Ant or Gradle.

To integrate JaCoCo with Ant, one needs to wrap the <junit> task with <jacoco:coverage>. In MPS the Ant tasks are generated by the build language so to integrate JaCoCo with MPS we need to extend the build language.

I have contributed such an extension to the mps-qa project. It is contained in the language org.mpsqa.testcov.buildIntegration.jacoco. The entry point is one of two build aspects that you can add to your MPS build scripts:

  • “test modules with coverage” (BuildAspect_MpsTestModulesWithCoverage) if you want to specify which tests to run, or
  • “run … with coverage” (BuildAspect_IndirectTestModulesWithCoverage) if you want to take the tests to run from another test configuration.

Here is an example listing the modules explicitly:

In both cases we also must specify which modules should have their coverage measured in the report (the “report coverage of” part).

Both aspects generate an Ant task named coverage.[coverage-configuration-name], optionally adding a dependency from module-tests task on this task (coverage will then also run as part of check or test targets).

The generated Ant XML is very similar to that generated by the standard “test modules” of MPS (BuildAspect_MpsTestModules), except that the <junit> task is wrapped with <jacoco:coverage> to measure the coverage and a <jacoco:report> task is appended to generate the reports.

Reports (HTML and XML) are placed under build/jacoco-reports/[build-script-name]/[coverage-configuration-name].

I have prepared a sample project that you can clone from GitHub and build using Gradle. I have also uploaded to GitHub Pages the HTML and XML reports generated for this sample by JaCoCo so you can get an idea of what information they contain without having to clone and build the project.

JaCoCo will only show the generated Java sources in the report because it does not have any knowledge of the MPS nodes. The mps-qa project already contains some rudimentary code to analyze JaCoCo measurement files (jacoco.exec) but it cannot yet be used to load arbitrary files from the file system. I expect this support to improve in the future, then we should be able to “lift” the coverage information into MPS and view coverage measurements on the level of model nodes rather than the generated code.