Friday, March 23, 2007

Grails vs. Rails Benchmark

This article: http://thoughtmining.blogspot.com/2007/03/where-are-all-groovy-andor-grails.html

Prompted me to do this:
http://docs.codehaus.org/display/GRAILS/Grails+vs+Rails+Benchmark

Feedback welcome.

I must stress this is not meant to be a flame war inciting benchmark, it has been done merely to allay the aforementioned bloggers fear that he expresses in this comment:

"Surely someone close to the respective projects has asked these hard questions of themselves?

Because I know the previous paragraph has to, has to, be true, then I can only assume one thing: someone has done this as an exercise...and the results were depressing. Why else would the results not be published? If Groovy or Grails was faster, even slightly so, then you can bet it would trumpeted far and wide. The fact that it's not leaves an unsettling feeling."

9 comments:

Stephan.Schmidt said...

Well, as a Grails enthusiast:

Rails isn't very fast and will get faster with a better VM. So this isn't very good looking for Grails I think.

On the other hand, there has been no optimization in Grails, the database is a limiting factor for both, scalability as mentioned with Terracotta should be good (could this move into Grails? :-)

What this test achieved though is that people don't need to be afraid when moving from Rails to Grails or when starting Grails development (I never was :-)

BUT if you want to be fast on Grails, why use MySQL? The fastest option for Java websites should be to use a Java DB (an option not available for Rails, at least if you don't use JRuby) like HSQL, Derby or (the fastest?) H2.

A test with H2 embedded should be really interesting and involves only changing the JDBC connection url :-)

Stephan.Schmidt said...

I forgot to add http://www.h2database.com/

(H2 being 5-10 times faster embedded than MySQL client/server)

james_027 said...

How is an embedded db differ from a client/server?

Anonymous said...

Embedded is usually faster because there is no need for TCP/IP communication with the database server. Instead the database server process (or thread) lives inside the web server, in this case, and the communication happens using regular function calls.

Paul Barry said...

I posted this over on Marc Palmer's blog, but I thought I cross post it here to make sure you can see it:

A rails application is not threaded like a Java app. What this means is that when you only have 1 mongrel process, all requests must be processed serially. This is not true for 1 tomcat instance. Since tomcat is a java app, it is threaded and process multiple requests at once, when gives it better performance under a concurrent load.

The way you handle more concurrent requests with a Rails app is to run multiple mongrel instances (on the same server, a different port for each) using mongrel_cluster. Then you use something like Apache 2.2’s mod_proxy_balancer to load balance the requests across the multiple mongrel instances.

It would be interesting to see these tests run again with mongrel_cluster running 2, 4, 6, 8, et.c instances to see what impact that would have. for this kind of test, I would guess that it actually wouldn’t be a tremendous boost, because each requests should be very fast. But when testing requests that can take longer than a few milliseconds, especially where uploading or downloading of large amount of data is concerned, the performance difference would probably be huge with using multiple mongrel instances.

Graeme Rocher said...

Hi Paul,

I've taken the advice of the Rails community and benchmarked with 10 mongrel instances and Pound as a load balancer.

I also configured Grails to have only 10 tomcat threads. However, in my tests Rails' performance degraded for most tests and Grails' improved.

Graeme Rocher said...

The updated benchmarks are here:

http://docs.codehaus.org/display/GRAILS/Grails+vs+Rails+Benchmark

Cheers

Anonymous said...

I'd suggest you run with nginx or mod_proxy_balancer or lighttpd with fair balance. This may not yield any better results, but few people suggest using pound now days.

Your benchmarks looks interesting and wonderful, but Ruby and Rails still do everything I ask them to, on any scale; I'll stick with Ruby. If speed was a problem for me and Rails, I'd appreciate this benchmark more.

Julian Klappenbach said...

Saying that performance is hindered by a full featured database is a bit disingenuous, as real production environments are not going to use in memory datastores. However, the commentary should also include the fact that Grails should be able to take advantage of the 2nd level caches implemented by the top JPA contenders, including Hibernate and Toplink. These caches not only provide read optimization for a single node, but are designed to be distributed throughout a cluster. The latter is more significant when considering that horizontal scalability is the key to a solid, load tolerant, highly available environment.

The fact that Grails stands on the backs of such software components as Hibernate, Swing, and Quartz is a pedigree that deserves serious consideration. Each of these projects, on their own, represent years of R&D and an evolution over nearly a decade to deliver best-of-breed.

Honestly, I thank Ruby on Rails and 37 signals for showing how unnecessarily complicated the lives of application developers have been. But in terms of tried and true solutions, it's hard to ignore the experience of Java. And that's precisely what Groovy and Grails allows us to capitalize.

I'm not trying to suggest that Ruby on Rails can't someday meet the stringent requirements set by large scale enterprise applications, but Java is here and now.