Using cobertura with gradle

I wanted to get code coverage for my current groovy project. Since I use gradle and all of my source is written in groovy, I searched for a gradle plugin that provides cobertura functionality for groovy files. I found two (perhaps there are a more available)

• Eric Wendelin’s Cobertura plugin
• Steve Saliman’s Cobertura plugin

Using both plugins wasn’t a big deal. There is a slight difference how to set them up in your build.gradle file though. Both plugins provide very similar configuration options to tweak cobertura’s behavior. Let’s take a look.

Using Eric Wendelin’s Cobertura plugin

Adding the plugin to the build.gradle

I’m using gradle 1.4 for this exercise and I added the following to my build.gradle:

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath "net.saliman:gradle-cobertura-plugin:1.1.0"
    }
}

apply plugin: 'cobertura

This will provide you with four new tasks:

• coberturaInstrumentMain
• coberturaInstrumentTest
• testCoberturaReport
• coberturaConfigureTest

So, which task to call to determine the code coverage? Simply run the test tasks and your source will be instrumented prior to executing your tests.

But what you really want, is to run the check tasks because this will not only run your tests on your previously instrumented source, but it will also create the coverage report. And that’s basically what we want.

The plugin provides some configuration options to tweak cobertura’s behavior

option	description
format	’html’ (default) or ’xml’
reportsDir	Path to report directory for coverage report. Defaults to ${project.reportsDir.path}/cobertura
includes	List glob paths to be reported on. Defaults to [‘**/*.java', ‘**/*.groovy', ‘**/*.scala’]
excludes	List glob paths to exclude from reporting. Defaults to [‘**/*Test.java', '**/*Test.groovy', '**/*Test.scala’]
ignores	List regexes of classes to exclude from instrumentation. Defaults to [‘org.apache.tools.*', 'net.sourceforge.cobertura.*’]

To specify the output format of cobertura’s coverage report you have to do this:

cobertura {
    format = ‘xml’
}

Using Steve Saliman’s Cobertura plugin

This plugin is based on the work of gradle_cobertura. The original version had some issue with the latest gradle version, so I decided to pick one of the 20 forks. The one with the most recent changes was Steve Saliman’s Cobertura plugin. You can find the setup instructions in the README. Currently the compatibility chart shows that gradle 1.1 is the highest supported version right now. I’m using it with gradle 1.2, 1.3 and 1.4 and they work too. So, let’s move ahead and give it a try.

Adding the plugin to the build.gradle

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath "net.saliman:gradle-cobertura-plugin:1.1.1"
    }
}
apply plugin: 'cobertura'

This will add 2 new tasks:

• cobertura
• instrument

To get your coverage report you need to run the cobertura task and the report will be created.

There are a couple of options available to this plugin to change the cobertura behavior:

option	description
coverageDirs	Directories under the base directory containing classes to be instrumented. Defaults to ${project.sourceSets.main.classesDir.path}
coverageDatafile	Path to data file to use for Cobertura. Defaults to ${project.buildDir.path}/cobertura/cobertura.ser
coverageReportDir	Path to report directory for coverage report. Defaults to ${project.reportsDir.path}/cobertura
coverageFormats	Formats of cobertura report. Default is a single report in ‘html’ format
coverageSourceDirs	Directories of source files to use. Defaults to ${project.sourceSets.main.java.srcDirs]
coverageIncludes	List of include patterns
coverageExcludes	List of exclude patterns
coverageIgnores	List of ignore patterns

Fix broken links in coverage report

In order to have a working link to the groovy source files in the coverage report, it’s necessary to add the following to your build.gradle. This will add the groovy source directory to the list of coverageSourceDirs

cobertura {
    coverageSourceDirs << project.sourceSets.main.groovy.srcDirs
}

Report location

For both plugins, the generated report can be found in build/reports/cobertura.

Summary

I have to admit, that both plugins are too similar in functionality to declare a clear favorite. In general, both plugins have subtle differences in regards of their configuration. I like that Eric Wendelin’s Cobertura plugin supports groovy (and scala) files right out of the box without further configuration. But this is only a very small advantage. I’ll stick with Steve Saliman’s Cobertura plugin for a while, since there is still active development. Not a strong reason, but as I mentioned earlier, there are only subtle differences between both plugins.

The choice is up to you.

Grape: adding a repo to the ivy configuration

Grape is a great thing. It’s one of my favorite features in groovy. It makes groovy script distribution so easy and painless. You want to use your groovy script on a new machine? Install groovy (if not already done) on the other machine and copy the script to the new machine. The script and all declared dependencies will be downloaded for you and the script is ready to run. What a timesaver.

@Grab(group='mycompany.com', module='awesome-lib', version='1.2')

This statement specifies a dependency to a library that your groovy script depends on. No need to copy this jar and all transitive dependencies manually to your machine. No need to make it manually available in the classpath. You know maven, gradle and ivy. They do the same job during build time. Grape provides the same dependency resolving functionality in the runtime scenario. No worries about missing dependencies anymore.

What happens if a required dependency is part of a not-so-famous or a company internal repository? Grape has a solution for it! You specify the repository with a Grape annotation and tell the dependency resolver to consult your company repository in order to resolve dependencies.

The following statement will add a repository named ‘my_unknown_repository’ and a root URL ‘http://repo.mycompany.com’ to the list of resolvers.

@GrabResolver(name='my_unknown_repository', root='http://repo.mycompany.com')

In case that you have a company internal repository (which also acts as a proxy to the outside world) and you always want to consult this repository without always remembering the GrabResolver annotation for every script you write, there is another way of specifying the repository. This option is less flexible, but it saves this one line in every script and probably makes more sense in a larger context like companies. To add your repository permanently to the resolver list, I recommend to obtain a copy of the configuration file that Grape uses to configure the underlying ivy to perform the heavy-lifting.

Grape uses a specific configuration that can be found in the groovy-x.y.z.jar. In order to customize the configuration, we need to create a grapeConfig.xml and add our repository there. But how does the content of the file look like? To preserve the existing configuration, we get a copy of the original config file from the groovy.jar.

Let’s jump to the command line (I assume that java’s jar is in the search path and /path/to/groovy exists):

jar -xf /path/to/groovy/lib/groovy-1.8.6.jar groovy/grape/defaultGrapeConfig.xml

This command extracts the defaultGrapeConfig.xml to the current directory. Open this file with your favorite editor and you will probably see something like this:

<ivysettings>
  <settings defaultResolver="downloadGrapes"/>
  <resolvers>
    <chain name="downloadGrapes">
      <filesystem name="cachedGrapes">
        <ivy pattern="${user.home}/.groovy/grapes/[organisation]/[module]/ivy-[revision].xml"/>
        <artifact pattern="${user.home}/.groovy/grapes/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/>
      </filesystem>
        <!-- todo add 'endorsed groovy extensions' resolver here -->
      <ibiblio name="codehaus" root="http://repository.codehaus.org/"   m2compatible="true"/>
      <ibiblio name="ibiblio" m2compatible="true"/>
      <ibiblio name="java.net2" root="http://download.java.net/maven/2/" m2compatible="true"/>
    </chain>
  </resolvers>
</ivysettings>

Now go ahead and copy one of the <ibiblio… /> lines and add your repository configuration. Finished with editing? Save the file as grapeConfig.xml in ~/.groovy/. From now on Grape will search your repository every time there is a dependency to resolve. If you want Grape to use only your repository, because it’s caching downloaded dependencies, you should remove all <ibiblio… /> lines, except the one for your repository.

Additional documentation about Grape can be found here.

What's so difficult, dude?

I have no idea!

How many times have you asked yourself this question? In the last couple of days I asked this question numerous times.
I'm currently working on some enhancements for a gradle plugin to simplify Flex development. This feels so slow and error prone, even if the task is not really complicated and it's not overly complex. But I'm stuck somewhere in the tools.

First: I'm no expert for IntelliJ! Not yet!
Since I used to work the last 10 years with Eclipse and I was very fluent with keyboard shortcuts, everything feels so different and complicated.
What is the shortcut to open all Implementors of this Interface in IntelliJ? I don't know yet. But this gets better from day to day. I found this nifty functionality triggered by Cmd+Shift+A. This will bring up a Popup window where you can search for an action and it shows the keyboard shortcut.

Second: The other unknown to my equation is gradle itself
It's not always obvious how the things are related and wired together. Don't get me wrong: Gradle is a great tool, but developing a plugin for gradle that does a little bit more than 'Hello Build', and finding the right pieces to use, is not always that easy. What makes it more complicated than necessary, is the fact that my dev env setup is probably not optimal to make this tasks easy.
Having two different projects (gradle and GradleFx) and trying to debug something in the latest gradle snapshot, is not really working for me. For that reason I started to write some unit tests, in order to increase my confidence level about my written code. But this only worked to a certain extend. Now I'm stuck in writing some, IMHO, basic unit tests and I can't do the simplest things without digging too deep into the internals of gradle.

Third: Probably it always too late for writing code.

Let's see what tomorrow brings.

List all jenkins jobs with a perforce scm configuration

Here is an instruction on how to execute this groovy script on your jenkins instance to display all jobs that use perforce as SCM provider. This was one of the first scripts I wrote to get an idea about the jenkins CLI and how to execute custom scripts. This script can be used to examine your perforce configurations used for your jobs. In my case it was the foundation to change all passwords for certain perforce user.

Prerequisites
You need to have a java installation on your machine and java must be available on your PATH. groovy itself is not required to execute the scripts.
I assume that you have Jenkins running on your machine, so that I refer to localhost to access the Jenkins instance.

1. If you already have a copy of jenkins-cli.jar on your machine, then skip to step 2. Otherwise open a browser and navigate to this URL http://localhost:8080/cli. Follow the instructions and download the jenkins-cli.jar to a known directory
2. Open a terminal or command window and change into the directory where your jenkins-cli.jar is located
3. Type java -jar jenkins-cli.jar -s http://localhost:8080 help. This will output a list of all available commands that this jenkins instance provides.
4. Okay, now go ahead and save this snippet as listAllProjectsWithPerforceSCM.groovy to the same directory where you previously downloaded the jenkins-cli.jar
   
5. Now enter the following on the command line:
   java -jar jenkins-cli.jar -s http://localhost:8080 groovy listAllProjectsWithPerforceSCM.groovy

This will show you something similar to this output:

Job 'Test' uses the following perforce configuration
------------------------------------------------------------------------
P4Port: localhost:1666
P4Client: testbuilder
P4User: testbuilder
P4Password: 0f0kqlwaDeXrEj0PA0z/+IXZM1f8G8QsgBlUgnUv8bbR2bzXLfa3AlrK8xqw==

That's it. There is nothing complicated about this script, but it shows some of the capabilities to automate certain tasks. Take the script as example to play around with it and explore new ways to interact with jenkins.

Thanks for reading this post.

ConfigSluper, ConfigObject and some stupid bugs

In my last project I wrote some scripts to handle the automatic integration of libraries from different Version Control Systems into our source code repository.
I wrote a script to generate an XML file, that served as input for an existing perl script.

To generate the XML file I used a groovy script which described the dependent components and the revision as well as the platform dependent location in the VCS (which is omitted here).



Processing this file is a pretty straightforward task.



The reason why I'm writing this up, is to tell that I spent a good amount of time, figuring out if there is a bug in the underlying ConfigObject. The ConfigObject is created by ConfigSluper().parse(...) and represents the data in-memory. Since ConfigObject inherits from LinkedHashMap, one can assume (and I really did!), that the semantics are like using a HashMap. But I was wrong! There happens some _magic_, when you do the following:



In my script there was a piece of code that relied on the sub-node count of the Component node. At a certain point, I was really convinced that I found a bug. But that would have been too obvious IMHO, that this kind of misbehavior had slipped through all testcases.
To make a long story short. I did some investigation and found the reason in the implementation of the ConfigObject


You can find the secret (if you will) in line 8. If you access a key in the ConfigObject that doesn't exist, then there will be an empty one created on the fly. This is not really bad, as long as you don't rely on the amount of nodes before and after querying the ConfigObject.

Create ProGit epub from source

A couple a days ago, I discovered that the Book Pro Git (Expert's Voice in Software Development) is available in source form on GitHub. Browsing through the repository, I found out that there is a folder called "epub" which contains a groovy script to create an epub Version of this book. Wow, that is really great! Using groovy and some other tools to create the epub. I started to clone the repository to create my own epub version of the book.


The first run of the script didn't work well on my Mac. To be honest it failed because of the missing dependencies (markdown and the path to calibre). I didn't want to download markdown and install it for this one time use. I decided to have a look into the script and find a different solution.


The creation of the epub is basically running in three steps:

1. Convert all Files from the given language directory and fix the links to the figures. Convert the files afterwards into HTML by using markdown. Write the file to the target directory
   
2. Merge all files from the target directory into one big HTML file
   
3. Call calibre's ebook-convert and create the epub out of it
   

I made some modifications to the original groovy script

• Use of markdownj instead the perl markdown implementation. I'm using groovy's @Grab annotation here to download the library on the first run of the script.
  
• Make use of groovy's great Regex support and do a one-line search and replace on the whole file content
  
• Added a custom CSS for the epub, to improve the layout of lists in the epub
  
• Reduced the overall LOC by removing unnecessary code
  

Now the solution looks like this (it's not perfect, but it works fine for me)

Running the script created now the epub that I can now read on my nook.

You can find the new version here.

One last thing to say: Even if this book has been made available freely by the Author and the Publisher, I recommend buying the printed version to support this kind of generosity. Let the epub Version of this great book be wonderful extra of the printed Book.

groovy oneliner - delete all files except those with a given extension (in this case *.tex)

Since LaTeX creates a lot of intermediary files, I use the following oneliner in groovy to wipe out all these files except the original LaTeX source:

Never use System.exit(1) in a groovy script

I wrote a small groovy script to to do some maintenance tasks on my machine. At some point the logic got a little bit more complicated and I decided to write some simple testcases. I executed the testscript and was really surprised about the output: the test should fail but it didn‘t! Now I tried to nail it down with limiting the scope to a very simple method call but the result was exactly the same. Hm, I thought it must have something todo with log4j that I used to write everything into the logfile. So I removed the log4j dependency and started the tests again. To my surprise the result didn‘t change at all. There was no output from junit on the console. I started to look a little bit closer at the class and walked through the constructor call to the method that I called during my test. Bang! There is was! In the constructor I checked for the existence of a certain resource (in this case a mounted volume). If the volume wasn‘t mounted I terminated the script with System.exit(1). I guess this was a really bad idea. Now I changed it to throw new RuntimeException(‘Volume not mounted‘) and everything is fine again.
Lesson learned: Never call System.exit(1) in constructor calls!