Pages

Friday, May 02, 2008

Comparing JUnit test results between CDK trunk/ and a branch #2

I reported earlier on how to compare unit test results between CDK trunk and a branch. Later, I noted that the diff typically overestimates the fail count, when unit tests had been moved to a different module. Therefore, a sort has to be added. The code is also updated for the SVN directory restructuring:
$ cdk cdk/
$ cdk trunk/
$ ant -lib develjar/junit-4.3.1.jar -logfile ant.log test-all
$ cd ../branches/miguelrojasch-CMLReact
$ ant -lib develjar/junit-4.3.1.jar -logfile ant.log test-all
$ cd ..
$ grep Testcase trunk/reports/*.txt | cut -d':' -f2,3 | sort > trunk.results
$ grep Testcase branches/miguelrojasch-CMLReact/reports/*.txt | cut -d':' -f2,3 | sort > branch.results
$ diff -u trunk.results branch.results
Obviously, you can still use wc for counting changes:
$ diff -u trunk.results branch.results | grep "^-Testcase" | wc -l
$ diff -u trunk.results branch.results | grep "^+Testcase" | wc -l
A second improvement, would be taking advantage of the ant.log files that are created anyway. Using the BeanShell tool tools/extractTestStats.bsh revision 10760 (see also this blog on bsh):
$ bsh trunk/tools/extractTestStats.bsh trunk/ant.log | grep run | grep -v total | grep -v antlogFile | cut -d' ' -f1-4 | sort > trunk.overview
$ bsh trunk/tools/extractTestStats.bsh branches/miguelrojasch-CMLReact/ant.log | grep run | grep -v total | grep -v antlogFile | cut -d' ' -f1-4 | sort > branch.overview
$ diff -u trunk.overview branch.overview