Wednesday, December 6, 2006

JRake, Part 1: Compiling

For as long as I've been working with Java, I've been in search of a build tool that didn't drive me, and those around me, bat-shit crazy. I've come to realize I have two somewhat conflicting requirements for such a tool:

  1. Must be based on a scripting language

    Builds get complicated, and I need a tool that will let me do arbitrarily complex things, preferably in a well-known scripting language with lots of supporting libraries.

  2. Must run on the Java platform

    The Java virtual machine takes time to start up. Since most tools that deal with Java code (e.g. javac) are themselves written in Java, this means there is a rather annoying interval between when you invoke them, and when they start doing actual work. This delay may seem insignificant, but on a large project where a build consists of many calls to many tools and gets run many times a day, all that wasted time can really harsh your flow.

    The solution is for the build tool to treat all of the supporting tools as libraries, not command line applications. For example, instead of calling javac directly, just invoke the method[]). That way you only incur the VM startup time once.

The tools I've used in the past don't meet these requirements, of course. Specifically:

  • make fails on both counts (along with most other tools).

  • ant fails on the first count, as it's not based an existing scripting language. While it's true that you can drop down to Java and do complicated things by writing your own tasks, the context switch is pretty jarring. One problem is that you have the extra step of compiling your tasks before you compile your code - and therefore your build script now needs a build script. But the real issue is that Ant tasks only let you put custom logic at the task level, not at the level that actually manages the tasks. For example, if you want to define some dependencies between projects in one place, and use that to drive a bunch of build steps for each project, you're in for a tough time. It can be done (we used XSLT to generate ant scripts on one project) but it ain't pretty.

  • rake, running on top of standard C-Ruby, fails to meet the second requirement. However, Rake itself is a damn nice tool. See Martin Fowler's article for a good summary.

Rake and JRuby

The solution, obviously, is to run Rake on top of JRuby (I will henceforth call this two-headed monster "JRake"). That way we have the power of Ruby at our disposal, with all the snappy goodness of a JVM based tool. The trick is to get JRake to invoke the main javac class directly, without starting a new VM. This actually isn't much of a trick, since integrating with Java is exactly what JRuby was designed for.

And so, after a bit of trial and error, here is a rakefile that does just that. It compiles any out-of-date java files in the "src" directory and puts the resulting class files in the "tmp" directory:

require 'java'

task :default => :compile

task :compile do
src_dir = 'src'
dest_dir = 'tmp'

Dir::mkdir(dest_dir) unless File::exist?(dest_dir)

javac(src_dir, dest_dir)

def javac(src_dir, dest_dir)
java_files = get_out_of_date_files(src_dir, dest_dir)

unless java_files.empty?
print "compiling #{java_files.size} java file(s)..."

args = ['-d', dest_dir, *java_files]

buf =
to_java_array(java.lang.String, args), != 0
print "FAILED\n\n"
print buf.to_s
print "\n"
fail 'Compile failed'
print "done\n"

def get_out_of_date_files(src_dir, dest_dir)
java_files = []
FileList["#{src_dir}/**/*.java"].each do |java_file|
class_file = dest_dir + java_file[src_dir.length,
java_file.length - src_dir.length - '.java'.length] + '.class'

# todo: figure out why File.ctime doesn't work
unless File.exist?(class_file)
&& >
java_files << java_file
return java_files

def to_java_array(element_type, ruby_array)
java_array = java.lang.reflect.Array.newInstance(element_type, ruby_array.size)
ruby_array.each_index { |i| java_array[i] = ruby_array[i] }
return java_array

If you want to play around around with this script but can't be bothered to install JRuby, Rake, and the other bits, I've created a complete "Hello World" project that includes everything you need. You can get it from subversion here:


Just checkout the project, make sure JAVA_HOME points to your JDK, and run the "build" script to see it go.

Next Steps

I've gotten JUnit and a few other cool things working with JRake, which I'll write about in my next post.

No comments: