Tuesday, March 13, 2007

JRuby, Groovy & Java Integration

As a follow-up to yesterdays article, I would like to point out the following article written by Ola Bini on the JRuby team: JRuby Regular Expressions.

This article represents everything that differs in the mentality between the Groovy & the JRuby teams. In essence, the JRuby team are planning on ditching java.util.regex and implementing their own regex parser to match the Ruby regex semantics.

Regardless as to whether this is a good thing or not, and regardless as to whether you can change the default implementation this is the essence of the debate over Java integration. So whenever you drop out of Java and into Ruby-land you need to remember that the semantics differ between regex implementations. To be clear: Java integration means more than just being able to call a method on a Java class.

To quote Dierk Koenig, author of Groovy in Action, who posted this comment yesterday:

"I would like to add that there is a fundamental difference between the two types of languages for the Java platform: type A, which are languages designed specifically for that platform (Groovy, Scala, etc.), and type B, which are ports of a foreign language (Jython, JRuby, Kawa, Bistro, and the likes).

While type B languages can achieve a decent level of interoperability, only type A languages can truly integrate with Java. This is because foreign languages not only have their own libraries, APIs, and such but more important, they come with their own object model, threading architecture, security concept, performance and debugging hooks, and last not least their own runtime architecture that can be fundamentally different from Java. Any type B language has to overcome this impedance mismatch breaking either Java's or the foreign language's behavior.

The acid test for integration is to replace any given Java class (be it abstract, or in the middle of an inheritance chain, or having overloaded methods, or using annotations) and replace it with an implementation in your language. This is what only type A languages can possibly achieve."

In other words, this is another example of how the JRuby developers have to make a hard choice in order to support C Ruby compatibility. Unfortunately on this occasion, it is Java integration that suffers.

In Groovy the expression ~/pattern/ results in an instance of java.util.regex.Pattern therefore the semantics are the same, you can call all the regular JDK methods on the object and the object can be passed to a Java object to hook into other frameworks and tools.

10 comments:

Charles Oliver Nutter said...

Oh come on Graeme, you should know better than this. I bit my tongue after Dierk's post yesterday, but I've got to comment on this one.

Look, the whole "type A type B" stuff is nonsense. We could just as easily make the Ruby language work with all normal Java types, literals return Java objects, and basically do exactly what Groovy's done by providing a friendly face for Java's APIs. Hell, in a bunch of places we already do that, automatically making List expose [] and << operators and multiple other types behave like Ruby equivalents. There's nothing about Ruby that precludes us ultimately just using Java types under the covers, without a separate type system.

And this is actually what Groovy does. For most types coming into Groovy, you have a meta model that represents how Groovy sees that object. Though it may be a Java String, Groovy decorates it with a bunch of Groovy-specific behavior. Of course you have to learn new things in Groovy...the only difference is the old things still work.

The move to better support Ruby's regex logic does not break or hinder Java integration in any way, but it makes Ruby work like Ruby developers expect it to work. That's the big difference, and one we're proud of with JRuby: there's a very large and growing community of Rubyists out there not tied to the JVM and Java APIs that are doing amazing things. We're aiming to make straight-up Ruby mode work exactly like Rubyists expect, but we're not doing anything to break the potential for Ruby as a JVM language in the future. You can do both.

And let's not forget that Grails was inspired by Rails, and still trumpets that lineage as a badge of honor. You can't have your cake and eat it too...you Groovy and Grails guys need to decide if you like Ruby and Rails or don't like them, because you claim you're "just as good" on one hand and then "we're way better" on the other hand. Don't compliment and insult in the same breath.

I am extremely excited that we have the opportunity to support both the Ruby and Java worlds, and support them well. For the Ruby world, that means making JRuby work like they expect Ruby to work. For the Java world, that means exposing Ruby's best capabilities with as little boundary between Ruby-land and Java-land as possible. Both are works in progress. But the fact that it's Ruby and not Groovy doesn't mean we're somehow crippled in this process.

masukomi said...

"The acid test for integration is to replace any given Java class (be it abstract, or in the middle of an inheritance chain, or having overloaded methods, or using annotations) and replace it with an implementation in your language. This is what only type A languages can possibly achieve."

Maybe I'm being naive but I don't see how what Charles is doing with JRuby fails this test. I mean. The fact that you're not doing it in Java inherently means you're doing it differently. I see no reason a class that used a JRuby regexp wouldn't be able to replace one that used a Java regexp and I see no reason why you'd want to replace the Java regex implementation with a function-alike in another language. If you want the Java one use the Java one. If you want the JRuby one use that, but why in the world would you want to just repace either one with the other? And, assuming they're not directly interchangeable how does this mean the language doesn't integrate well?

Anonymous said...

Re: "how does this mean the language doesn't integrate well?"

The JRuby approach is equally valid to the Groovy approach but a little different in philosophy. JRuby should be making things easy for Ruby folk. That makes perfect sense. Groovy on the other hand is explicitly focusing on making Java integration work seamlessly. It's not that JRuby isn't also trying to do this but just one or priorities. You can't make a language have everything as top priority and Java integration is just one of Groovy's higher priority areas of focus.

As an example, if you have some existing Java code which works with Regex parameters, Groovy would aim to always be able to pass its notion of Regex to that existing code. You could have a different implementation or model of Regex's under the covers but then will it always work with the legacy code? In this particular case, Groovy avoids the problem by using the same implementation under the covers and provides useful syntactic sugar on top.

It's not a black and white issue though. If you want to sort things, using Groovy's Closure capabilities will usually lead to the nicest code. You can also implement Comparable and do it the traditional Java way too. This would give you the best Java integration. So, Groovy happily supports both. If you started implementing Comparable in JRuby on the other hand, Ruby people will think you are doing something strange and the infrastructure in the Ruby object model to handle Comparable just isn't there. This is not the end of the world. The closure way is likely to lead to nicer JRuby code but Groovy's way will lead to nicer integration.

Graeme Rocher said...

Charles, unfortunately your comments are incorrect. The type A/type B problem does exist. You "could" make JRuby operate with Java types under the covers, but then you would break C Ruby compatibility.

This is exactly the problem you have here with regex and having other stuff working IS important because it provides a migration and learning path for newcomers to the language.

As for whether we love Ruby/Rails, of course we do, a lot the inspiration for Groovy comes from the Ruby community just as the inspiration for builders comes from the Groovy community. I've said before the Grails naming is unfortunate and in hindsight we should have chosen a different name, nevertheless the brand works for us now.

As for your original post, I think you need to be a little more diplomatic when talking about Groovy. Saying things like "Groovy adds no value to Grails" and "anything Groovy can do Ruby can do better" is not going to win you any friends.

Charles Oliver Nutter said...

graeme: Now let's not start misquoting :) I said the following, not the quotes you provided:

"...it's not the Groovy part [of Grails] that's most interesting here ..."

and

"...everything you can do in Groovy you can do in Ruby (plus more)."

Never did I say Groovy adds nothing to Grails, I only believe that the non-Groovy part of Grails is much, much more interesting than the fact that it's fronted with Groovy at present.

And I never said Ruby does what Groovy does "better", I said it can do what Groovy does and "more". There's a subtle difference there, but I don't think many folks would argue when taken solely as languages (excluding the muddy waters of API support and Java integration) that somehow Groovy is more flexible than Ruby. I don't think many other language developers would try to make that claim either. That's what makes Ruby so interesting for these kinds of problems, and so difficult to implement well.

Ultimately, though, my only purpose with both comments was to say that "everything Groovy does in Grails could be done with Ruby", to which you agreed in your response post (or I took it as agreement, anyway). It was never, ever a debate about language A versus language B, it was a treatise on the future of Grails.

On the supposed type A/type B problem...JRuby's array is just implemented as a Java array, and it implements all appropriate Java collection interfaces. Hash is implemented with HashMap. String is a byte[] internally, but there's nothing stopping it from being used as a CharSequence. Fixnum is a Long. Bignum is a BigInteger. Float is a Double, and so on. We're more like Groovy in that regard than you think, though we provide more Ruby-like semantics for those types and their interactions. And we have been knocking down more and more barriers between the two worlds.

We're also looking at making the next step soon: adopting a meta model similar to Groovy, but with Ruby semantics. Under the covers all objects would basically be Java types...String, StringBuffer, File, whatever. But they'd have Ruby semantics applied to them, just like in Groovy. And what would change for Ruby developers? Practically nothing...they'd still see a Ruby view of things, even though we'd pass Java types around. This is real, and it can work. It works for Groovy, doesn't it?

On Builders: Builder syntaxes have been around as long as Lisp has been around, so I don't think it's quite fair to claim that Groovy started that. Groovy was probably the first one to call them builders, though.

Breaking one world or the other: In the end there's no reason we can't support both modes of execution, with Ruby being used in some cases exactly as the C implementation is used and in other cases used as "just another JVM language." It's somewhat challenging, but it's far from impossible. Of course nobody expects that you're going to take Ruby code calling Java APIs and run it on the C impl, just like folks calling C code don't expect to run in JRuby. So there *are* two modes of execution being supported today, and even now JRuby users can choose to run with only pure Ruby or to make heavy use Java APIs. They make a conscious decision to move into "Java-only Ruby" and at that point we can support them with more Java-friendly semantics. It isn't rocket science.

Now of course you're always going to love your language of choice, and I'm going to love mine. And we'll fight to show that our favorite language is on par with others. But let's always keep it civil, avoid misquoting each other, and try to present the facts fairly. :)

Anonymous said...

Hmmm Charles, I'm curious about what "non-Groovy part of Grails" you find interesting, as we know it uses Spring, Hibernate, SiteMesh & Quartz... Remember that in Grails, Groovy is the glue for all the parts, whereas in Rails Ruby is the language/platform for everything.

Anonymous said...

Though I understand the differences of both approaches, as do both Graeme and Charles, I think the two parties are too close to argue dispassionately.

I have to agree with Graeme here. I love JRuby for what it is, but what it is not is what makes Groovy so special. The A/B discussion is actually quite appropriate when you get down to it, but probably not to the level that Graeme describes

However, Groovy is the clear and appropriate language if you wish to 100% integrate into a Java environment. Not only should this be quite obvious, we shouldn't discuss it anymore.

Anonymous said...

I posted this on the comments of the entry prior to this, and my apologies but I'm going to copy and paste the pertinent part(s) here too.

This is a classic case of when all you have is a hammer, everything becomes a nail. To steal a verse from the Bible, "all things are permissible, but not all things are beneficial."

Of course you can tweak this, move that, delete this, click that and get JRuby to run on Grails, but if you dilute the effort now into JRuby and Groovy support, what you're doing is halving the development resource commitment, doubling the documentation required, fracturing the learning curve and all for what purpose? With my experience of open source projects, I think Grails is at a very sensitive stage right now. If its simplicity and purity is retained and cherished now, I agree with Graeme that 2007 is going to be a great year for Grails. If you start complicating things now (or ever) I think Grails is headed for the bin of projects that had a great idea but lost the plot.

I don't get it. If people love Rails/Ruby so much, then damn well stick to Rails and Ruby. If scalability is your concern, fix that, but fix it in Rubyland. Stay the hell out of Javaland. What Grails takes from Rails is the model, the approach, the 'opinion'. It doesn't long for the Ruby language. Groovy (that name notwithstanding) is more than sufficient to glue together the parts.

I'm sorry for my ranting, but I'm so sick and tired of discovering a really cool, new technology, getting excited about it's application in my world of interest, only to see it fractured by the genuine but misguided efforts of some people who seem to take more pleasure in the framework than in what the framework can accomplish. I fall in the latter category, I make no secret of that.

There is a disturbance in The Force, and its name is JRuby. :-)

backspaces said...

For a while now, I've been working on using the agile jvm languages to drive an interesting java graphics library, Processing: http://processing.org. Why? Mainly because we wanted to cut to the chase and see what the trade offs were in these environments are, and how well they actually do integrate with java.

The systems we are currently looking at are:
- Jython
- Javascript (Rhino)
- Groovy
- JRuby

We first moved the Processing code out of its IDE and into Eclipse, then tried Jython. I've not written on the further tests, but all are done now except for a bug we need to work around in JRuby.

Boy have we learned a lot from such a simple experiment!

I will tell you that for us, performance is quite an issue. This could easily mean we're using the wrong approach entirely. So far we've basically taken a trivial program that jiggles 1000 rectangles within a window:
http://backspaces.net/models/RandBoxes/applet/
When these are all running well, we'll extend the test to a more interesting one:
http://backspaces.net/models/RoadGrid/applet/
.. but for now, we're focusing in the simpler RandBoxes.

The translations are literal .. we try as much as possible to mimic the original java. This is a bit awkward due to Processing's use of inner classes, but we've got a reasonable workaround for that. See the Eclipse and Jython articles at:
http://backspaces.net/

When running raw java, the speed in frames per second for java is around 170-200 fps, depending on our use of jdk 1.4 or 1.5, and whether or not we're using a local application or an applet in a browser. The times for the jdk languages are:
Jython: 16.5 fps
Groovy: 24.0 fps
Rhino: 30.f fps
Now as in any performance test, this is really raw data and we should not be surprised in dramatic improvements when we become more savvy in these environments. But as you can see, the gap is quite large.

I realize at this point its early days, but when we talk about java integration, I think we also need to talk about java performance. We're looking into things like piped MVC stunts (the agile language creates a controller and a model/data pipe/file, the view is pure Processing/java) which would likely be quite a bit faster. And you folks may want to consider that too: just how DO you want to integrate with java and maintain performance.

Thus far you've concentrated on systax/semantics. We need java performance!

Owen

Demon said...

Regular expression is really wonderful to parsing HTML or matching pattern. I use this a lot when i code. Actually when I learn any new langauge, first of all I first try whether it supports regex or not. I feel ezee when I found that.

http://icfun.blogspot.com/2008/04/ruby-regular-expression-handling.html

Here is about ruby regex. This was posted by me when I first learn ruby regex. So it will be helpfull for New coders.