Wednesday, December 6, 2006

JRake, Part 1: Compiling

For as long as I've been working with Java, I've been in search of a build tool that didn't drive me, and those around me, bat-shit crazy. I've come to realize I have two somewhat conflicting requirements for such a tool:

  1. Must be based on a scripting language

    Builds get complicated, and I need a tool that will let me do arbitrarily complex things, preferably in a well-known scripting language with lots of supporting libraries.

  2. Must run on the Java platform

    The Java virtual machine takes time to start up. Since most tools that deal with Java code (e.g. javac) are themselves written in Java, this means there is a rather annoying interval between when you invoke them, and when they start doing actual work. This delay may seem insignificant, but on a large project where a build consists of many calls to many tools and gets run many times a day, all that wasted time can really harsh your flow.

    The solution is for the build tool to treat all of the supporting tools as libraries, not command line applications. For example, instead of calling javac directly, just invoke the method com.sun.tools.javac.Main.main(String[]). That way you only incur the VM startup time once.

The tools I've used in the past don't meet these requirements, of course. Specifically:

  • make fails on both counts (along with most other tools).

  • ant fails on the first count, as it's not based an existing scripting language. While it's true that you can drop down to Java and do complicated things by writing your own tasks, the context switch is pretty jarring. One problem is that you have the extra step of compiling your tasks before you compile your code - and therefore your build script now needs a build script. But the real issue is that Ant tasks only let you put custom logic at the task level, not at the level that actually manages the tasks. For example, if you want to define some dependencies between projects in one place, and use that to drive a bunch of build steps for each project, you're in for a tough time. It can be done (we used XSLT to generate ant scripts on one project) but it ain't pretty.

  • rake, running on top of standard C-Ruby, fails to meet the second requirement. However, Rake itself is a damn nice tool. See Martin Fowler's article for a good summary.

Rake and JRuby

The solution, obviously, is to run Rake on top of JRuby (I will henceforth call this two-headed monster "JRake"). That way we have the power of Ruby at our disposal, with all the snappy goodness of a JVM based tool. The trick is to get JRake to invoke the main javac class directly, without starting a new VM. This actually isn't much of a trick, since integrating with Java is exactly what JRuby was designed for.

And so, after a bit of trial and error, here is a rakefile that does just that. It compiles any out-of-date java files in the "src" directory and puts the resulting class files in the "tmp" directory:

require 'java'

task :default => :compile

task :compile do
src_dir = 'src'
dest_dir = 'tmp'

Dir::mkdir(dest_dir) unless File::exist?(dest_dir)

javac(src_dir, dest_dir)
end

def javac(src_dir, dest_dir)
java_files = get_out_of_date_files(src_dir, dest_dir)

unless java_files.empty?
print "compiling #{java_files.size} java file(s)..."

args = ['-d', dest_dir, *java_files]

buf = java.io.StringWriter.new
if com.sun.tools.javac.Main.compile(
to_java_array(java.lang.String, args), java.io.PrintWriter.new(buf)) != 0
print "FAILED\n\n"
print buf.to_s
print "\n"
fail 'Compile failed'
end
print "done\n"
end
end

def get_out_of_date_files(src_dir, dest_dir)
java_files = []
FileList["#{src_dir}/**/*.java"].each do |java_file|
class_file = dest_dir + java_file[src_dir.length,
java_file.length - src_dir.length - '.java'.length] + '.class'

# todo: figure out why File.ctime doesn't work
unless File.exist?(class_file)
&& java.io.File.new(class_file).lastModified > java.io.File.new(java_file).lastModified
java_files << java_file
end
end
return java_files
end

def to_java_array(element_type, ruby_array)
java_array = java.lang.reflect.Array.newInstance(element_type, ruby_array.size)
ruby_array.each_index { |i| java_array[i] = ruby_array[i] }
return java_array
end

If you want to play around around with this script but can't be bothered to install JRuby, Rake, and the other bits, I've created a complete "Hello World" project that includes everything you need. You can get it from subversion here:

svn://svn.foemmel.com/blog/jrake/compiling

Just checkout the project, make sure JAVA_HOME points to your JDK, and run the "build" script to see it go.

Next Steps

I've gotten JUnit and a few other cool things working with JRake, which I'll write about in my next post.

No comments: