Saturday, March 24, 2007

Updated Grails vs Rails Benchmark

So after a couple of people pointed out my naivety in configuring Rails, I decided to re-run the tests. What I did was configure Rails with a 10 mongrel cluster and the Pound load balancer as per Jared's recommendation. However, to make things more equal I reduced the Grails Tomcat server's thread pool down to 10 by setting maxThreads=10 in Tomcat's server.xml.

The result was that Rails' performance degraded in all except the long running query test, whilst Grails' performance significantly improved in all except the same test. Clearly, since I have only dual core's on my MacBook giving Rails or Grails more processes doesn't necessarily improve things for the shorter tasks. Check out the updated benchmarks.

Again, I'm no Rails performance tuning wizard so if any Rails expert can suggest improvements to the Rails configuration please don't hesitate to shout.


Steve said...

Check your memory when running under Mongrel, if your system is starting to swap due to RAM requirements of so many Ruby processes, that will have hurt the benchmark.

In testing I've done in the past, I didn't see Mongrel slow down like that, rather I saw minimal increases.

My typical Ruby process size was 30-50MB which means you'll need 300-500MB RAM free so that virtual memory swapping doesn't hurt your benchmark.

Increasing threads doesn't increase RAM requirements like increasing processes does. Not that more threads don't eat more ram, it's just usually an insignificant proportion.

Anonymous said...

I believe rails runs a significant amount of unnecessary code in 'development' mode. Try -e production.

masukomi said...

I agree about running Rails in production mode for it to be an even comparison but I'm also really curious as to why Grails seemed to perform better under tomcat when you restricted it to 10 threads. I'm assuming the default is more than ten so... that just strikes me as really odd.

Graeme Rocher said...

I did configure rails in production mode, that was a copy-and-paste mistake from Jared's blog.

As for Grails & Tomcat, the more threads doesn't necessarily mean better performance on a machine with only dual cores.

Jochen "blackdrag" Theodorou said...

Maybe it would be really interesting to actually see the overhead produced by Groovy/Grails compared to JSP or something like that... maybe by doing a secret test ;)

I guess the increased performance for only 10 threads for grails is related to a synchronization problem in Groovy. There are some parts here that need optimization, especially the MetaClassRegistry, probably the GroovyClassLoader too. I think I also remember that a multiprocess application gets a penalty over a threaded application when run on a multicore CPU, I am not sure about this. I think it was something about the L2 cache, really not sure. A test on a machine with two real CPUs should show it, but on the other hand the future processor seems to be a multicore CPU. On the other hand future Ruby will maybe change the threading model too.

Then there is another thing about using multiple parallel processes... the general rule here is that you get an increased performance for each possible parallel process. But in real life these processes need to communicate with each other, and they need to be scheduled..

This communication overhead grows with the number of processes and it might happen, that due to this overhead, the overall performance becomes less then the performance with just a single process. I tried to find a picture showing that.. well just imagine the graph for sqrt(x)-x/100 It will grow fast, reach a maximum around 25 and then go down. Unlike this example a real case should never reach 0, it is really just to imagine what it looks like. There are surly better people to explain this ;)

Anyway, I guess it means that Mongrel is using too many processes, while Java is not yet in that state. Maybe the sated state is in Java above one thread and in Mongrel below one process.. I really don't know, running the tests with different amounts of processes/threads would show that.

Then there is one thing about java.util.Random. I am not sure about the performance of rnd in Ruby, but in Java the nextInt method is very slow compared to the rnd function known from many other applications. explains the reason, rnd is normally not doing such things. But hey, this is no scientific simulation and I guess the test depends not much on the performance of Random... which then tells me, that you maybe should replace it with other code?

The next problem with Random is, that used like this you won't get repeatable sequences. With the additional problem that I don't know how to get this working in Rails since all ruby processes don't share the same random number sequence offset, a simple use of srand won't solve that for Ruby then. Each process would use the same offset, repeating the same numbers. Letting them share that somehow would mean to have less performance for Ruby. On the other hand the Groovy solution uses a shared object here, but not thread safe. Isn't it funny how such a simple piece of code can lead to such problems?

Well, as I said, in the end I would replace that part with other code or keep it like it is. There is no real problem if the data does not really depend on the generated numbers.

oldmoe said...

I know I am too late to join the party but man have I been awed by the benchmark results!

Your machine was SWAPPING with the 10 mongrels + pound + whatever else you had running. This consumed your 1GB of Ram.

If you care to retry again then only run mongrels as your memory can handle (3 would be fine in your case)

And regarding steve's comment above mentioning minimal increases with increasing mongrels:
Unless there is some sort of IO congestion then your performance should increase almost linearly with mongrel instances up to the number of processor cores you have. From that point forward you will start to see minimal increases (depending on your IO usage patterns)

battery said...

[...]The offline scripting tool might add some relief here, but it’s still sort of a pain. I’ve had to separate “build” from “configuration” steps.[...]

ATField said...

runescape money
runescape gold
runescape money
runescape gold
buy runescape gold buy runescape money runescape items
runescape accounts
runescape gp
runescape money
runescape power leveling
runescape money
runescape gold
dofus kamas
cheap runescape money
cheap runescape gold
Guild Wars Gold
buy Guild Wars Gold
lotro gold
buy lotro gold
lotro gold
buy lotro gold
lotro gold
buy lotro gold

Hellgate Palladium
Hellgate London Palladium
Hellgate money
Tabula Rasa gold tabula rasa money
Tabula Rasa Credit
Tabula Rasa Credits
Hellgate gold
Hellgate London gold
wow power leveling
wow powerleveling
Warcraft PowerLeveling
Warcraft Power Leveling
World of Warcraft PowerLeveling World of Warcraft Power Leveling runescape power leveling
runescape powerleveling
eve isk
eve online isk
eve isk
eve online isk
tibia gold
Fiesta Silver
Fiesta Gold
Age of Conan Gold
buy Age of Conan Gold
aoc gold

china tour
beijing tour
beijing travel
china tour
tibet tour
tibet travel
computer monitoring software
employee monitoring

derfefww said...

I always heard something from my neighbor that he sometimes goes to the internet bar to play the game which will use him some hero gold,he usually can win a lot of hero online gold,then he let his friends all have some hero online money,his friends thank him very much for introducing them the hero money,they usually buy hero gold together.

coco0610 said...

handmade jewelry
jewelry wholesale
discount jewelry
handcrafted jewelry
cheap jewelry
crystal jewelry
wholesale discount jewelry
wholesale fashion jewelry
wholesale china jewelry
wholesale discount jewellry
china jewelry manufacturer
wholesale pearl
wholesale jewelry
costume jewelry
handmade jewelry
fashion jewelry
pearl jewelry

Anonymous said...

看房子,買房子,建商自售,自售,台北新成屋,台北豪宅,新成屋,豪宅,美髮儀器,美髮,儀器,髮型,EMBA,MBA,學位,EMBA,專業認證,認證課程,博士學位,DBA,PHD,在職進修,碩士學位,推廣教育,DBA,進修課程,碩士學位,網路廣告,關鍵字廣告,關鍵字,廣告,課程介紹,學分班,文憑,牛樟芝,段木,牛樟菇,日式料理, 台北居酒屋,燒肉,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,台北結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,住宿,訂房,HOTEL,飯店,造型系列,學位,牛樟芝,腦磷脂,磷脂絲胺酸,SEO,婚宴,捷運,學區,美髮,儀器,髮型,牛樟芝,腦磷脂,磷脂絲胺酸,看房子,買房子,建商自售,自售,房子,捷運,學區,台北新成屋,台北豪宅,新成屋,豪宅,學位,碩士學位,進修,在職進修, 課程,教育,學位,證照,mba,文憑,學分班,網路廣告,關鍵字廣告,關鍵字,SEO,关键词,网络广告,关键词广告,SEO,关键词,网络广告,关键词广告,SEO,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,居酒屋,燒烤,美髮,儀器,髮型,美髮,儀器,髮型,美髮,儀器,髮型,美髮,儀器,髮型,小套房,小套房,進修,在職進修,留學,證照,MBA,EMBA,留學,MBA,EMBA,留學,進修,在職進修,牛樟芝,段木,牛樟菇,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,住宿,民宿,飯宿,旅遊,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,美容,美髮,整形,造型,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,設計,室內設計,裝潢,房地產,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,進修,在職進修,MBA,EMBA,住宿,民宿,飯店,旅遊,美容,美髮,整形,造型,設計,室內設計,裝潢,房地產,進修,在職進修,MBA,EMBA

Anonymous said...




Anonymous said...

In my people mind, the knight noah is very expensive, but now I want to say that in here you can buy the cheap knight noah, it is really, from now on, you can come here to buy knight noah, it only spend a little money then can buy much knight gold.

Anonymous said...

^^ nice blog!! ^@^

徵信, 徵信網, 徵信社, 徵信社, 感情挽回, 婚姻挽回, 挽回婚姻, 挽回感情, 徵信, 徵信社, 徵信, 徵信, 捉姦, 徵信公司, 通姦, 通姦罪, 抓姦, 抓猴, 捉猴, 捉姦, 監聽, 調查跟蹤, 反跟蹤, 外遇問題, 徵信, 捉姦, 女人徵信, 女子徵信, 外遇問題, 女子徵信, 外遇, 徵信公司, 徵信網, 外遇蒐證, 抓姦, 抓猴, 捉猴, 調查跟蹤, 反跟蹤, 感情挽回, 挽回感情, 婚姻挽回, 挽回婚姻, 外遇沖開, 抓姦, 女子徵信, 外遇蒐證, 外遇, 通姦, 通姦罪, 贍養費, 徵信, 徵信社, 抓姦, 徵信, 徵信公司, 徵信社, 徵信公司, 徵信社, 徵信公司, 女人徵信,

徵信, 徵信網, 徵信社, 徵信網, 外遇, 徵信, 徵信社, 抓姦, 徵信, 女人徵信, 徵信社, 女人徵信社, 外遇, 抓姦, 徵信公司, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 女人徵信社, 徵信社, 徵信, 徵信社, 徵信, 女子徵信社, 女子徵信社, 女子徵信社, 女子徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信社,

徵信, 徵信社,徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 外遇, 抓姦, 離婚, 外遇,離婚,

徵信社,徵信, 徵信社, 徵信, 徵信社, 徵信,徵信社, 徵信社, 徵信, 外遇, 抓姦, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信社, 徵信社, 徵信社,徵信,徵信, 徵信, 外遇, 抓姦

cvxv said...

看房子,買房子,建商自售,自售,台北新成屋,台北豪宅,新成屋,豪宅,美髮儀器,美髮,儀器,髮型,EMBA,MBA,學位,EMBA,專業認證,認證課程,博士學位,DBA,PHD,在職進修,碩士學位,推廣教育,DBA,進修課程,碩士學位,網路廣告,關鍵字廣告,關鍵字,課程介紹,學分班,文憑,牛樟芝,段木,牛樟菇,日式料理, 台北居酒屋,日本料理,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,台北結婚,場地,住宿,訂房,HOTEL,飯店,造型系列,學位,SEO,婚宴,捷運,學區,美髮,儀器,髮型,看房子,買房子,建商自售,自售,房子,捷運,學區,台北新成屋,台北豪宅,新成屋,豪宅,學位,碩士學位,進修,在職進修, 課程,教育,學位,證照,mba,文憑,學分班,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,台北住宿,國內訂房,台北HOTEL,台北婚宴,飯店優惠,住宿,訂房,HOTEL,飯店,婚宴,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北結婚,場地,結婚,場地,推車飲茶,港式點心,尾牙春酒,台北結婚,婚宴場地,結婚,婚宴場地,推車飲茶,港式點心,尾牙春酒,台北結婚,場地,居酒屋,燒烤,美髮,儀器,髮型,美髮,儀器,髮型,美髮,儀器,髮型,美髮,儀器,髮型,小套房,小套房,進修,在職進修,留學,證照,MBA,EMBA,留學,MBA,EMBA,留學,進修,在職進修,牛樟芝,段木,牛樟菇,關鍵字排名,網路行銷,PMP,在職專班,研究所在職專班,碩士在職專班,PMP,證照,在職專班,研究所在職專班,碩士在職專班,SEO,廣告,關鍵字,關鍵字排名,網路行銷,網頁設計,網站設計,網站排名,搜尋引擎,網路廣告,SEO,廣告,關鍵字,關鍵字排名,網路行銷,網頁設計,網站設計,網站排名,搜尋引擎,網路廣告,SEO,廣告,關鍵字,關鍵字排名,網路行銷,網頁設計,網站設計,網站排名,搜尋引擎,網路廣告,SEO,廣告,關鍵字,關鍵字排名,網路行銷,網頁設計,網站設計,網站排名,搜尋引擎,網路廣告,EMBA,MBA,PMP
,在職進修,專案管理,出國留學,EMBA,MBA,PMP,在職進修,專案管理,出國留學,婚宴,婚宴,婚宴,婚宴,漢高資訊,漢高資訊,比利時,比利時聯合商學院,宜蘭民宿,台東民宿,澎湖民宿,墾丁民宿,花蓮民宿,SEO,找工作,汽車旅館,阿里山,日月潭,阿里山民宿,東森購物,momo購物台,pc home購物,購物