/ Ruby

The Perfect Cloud

A List of Non-English Ruby and Rails News Sites

When Ruby Inside started over two years ago, there were only a handful of sources for Ruby related news. The announcements on the ruby-talk mailing list (which Ruby Weekly News - now deceased - rounded up on the Web), del.icio.us, and a few popular Rubyists blogs (such as Why's Redhanded). Now, however, there are lots of options, including Ruby Inside itself, Rails Inside, RubyFlow, Ruby Reddit, and Planet Ruby on Rails.

But what about the Ruby and Rails news for non-English speakers? I've been keeping my eye on a number of foreign-language Ruby and Rails news sites, and want to highlight some of the best:

Russia / Россия / русско

RubyFlow.ru is a Russian equivalent of the English language RubyFlow.com. Like RubyFlow, it's a community blog where news about Ruby and Rails libraries, projects, tutorials, and so forth can be posted.

They've done an excellent job on the design - it rather makes me wish I could read Russian!

RubyMag is a new weekly summary of Ruby and Rails news for Russian speaking developers.

The Netherlands / Dutch / Nederlands

RubyWeb is a Dutch RubyFlow clone. There seem to be posts for most days, and it's primarily updated by "TomEric" of i76.

Ruby En Rails is a Dutch Rails-themed blog. It doesn't seem to be particularly well updated, but has news relating to Dutch Rails events, along with general tutorials written in Dutch.

China / 中国

Flow.RubyNow.com is a Chinese language equivalent of RubyFlow. It seems to mostly feature the same items as RubyFlow but in Chinese.

Japan / 日本 / 日本語

RubyFlow-ja is a Japanese language translation of RubyFlow, and the first such translation. It's maintained by Makota Kuwata - of Erubis fame. It features the most interesting of the RubyFlow stories (that is, not all of them).

The Rubyist is a long-time Japanese language online Ruby magazine. It has a very strong reputation in the Japanese Ruby community, but as the articles aren't translated into English, many Anglophone Rubyists are not aware of it. It's not to be confused with Jeremy McAnally's newer print-based The Rubyist magazine.

Indonesia

For some time, the official Indonesian Ruby site featured translations of choice Ruby Inside articles into Indonesian. This appears to have stopped recently, but it is still a key resource for Indonesian Rubyists.

Brazil / Brasil / Portuguese / Português

Ruby-Br.org is a key resource for Brazilian Ruby developers. It summarizes a lot of general Ruby news in Portuguese.

Akita on Rails is a Brazilian Rails-focused blog written by perhaps the most famous Brazilian Rubyist of all - Fabio Akita (who was the principal organizer of the successful Rails Summit Latin America conference recently).

Others? I Need Your Help!

I've been trying to find Ruby news sites in other popular languages, such as French and German, but have been drawing blanks. Where are French, German, Swedish, Danish, Spanish and Italian Ruby developers getting their news from? I want to turn this post into a resource for finding foreign language Ruby news sites, so please leave your links and suggestions in the comments.

Duby Presentation at Ruby Users of MN

Step by step guide to contributing code to Rails

Always wanted to bask in the glory of being a Rails contributor? Mike Gunderloy has compiled a 12-step program to get you there from square. It’s never to late to get started. According to some calculations, we’ve already had some 1,400 different people contribute.

Wide Finder 2: processing 42GB of httpd logs, 300X faster than naïve Ruby

The Wide Finder 2 benchmark measures the speed at which a program can analyze 42GB worth of webserver logs and generate basic statistics (top URLs by hits, byte count and 404 errors, top clients and referrers) on a pretty beefy machine: a 8-core Sun Fire T2000 with 32 hardware threads and 32GB of RAM.

Several people have written a multitude of implementations in various languages, including C , OCaml, C, Java, Scala, Groovy, Fan, Python, Perl, Ruby, and combinations thereof. The results are presented on this table.

Speedup

The reference implementation was written in Ruby and took over 25 hours to process the logs. The fastest versions, coded in C and OCaml, can do it in around 5 minutes, nearly 300 times faster.

In the case of OCaml, this speedup can be broken down into 3 factors.

Use of a more efficient language implementation: 1 order of magnitude

This does not necessarily cause an explosion in code size, as shown by the wf2_simple.ml semi-naïve port to OCaml, which is in fact shorter than the reference Ruby implementation by line count (this is far from being the case in C , though), but runs an order of magnitude faster.

It is interesting to analyze the bang-per-buck ratio for the different languages: while two C implementations ultimately yielded the highest performance (a third one failed to parallelize properly, and was over three time slower than the top C or OCaml versions) , they do so at the cost of an explosion in the amount of code, requiring 2 to 3 times more lines than the OCaml version whose performance was within 10% of the best one, even though the C entries used libraries like Boost.

Parallelism: 1 order of magnitude

The Wide Finder 2 problem is fairly easy to decompose into smaller subproblems (essentially by having several workers processing different parts of the logs). The T2K is a bit particular in that each of its 32 hardware threads (which are seen as 32 CPUs when you program) is much less powerful than the x64 cores we're using most of the time, so parallel execution is a must.

I parallelized the OCaml program trivially by using a semi-standard 15-line higher-order function which accepts a function and executes it in another process.

Small-scale optimizations that apply to both the single-threaded and the parallel cases: 2-4X

The Wide Finder 2 implies lots of IO activity, which proved to be relatively hard to optimize on the T2K, because it you can easily saturate a core by doing mere IO (i.e., a single core is barely able to cope with the sustained read rate of the disk). While my first OCaml implementations used line-oriented IO, like the reference version in Ruby, the fastest one (like most other entries) uses block-oriented IO, which accounted for a large part of the linecount increase.

Conclusions

Optimization potential

Scaling ActiveRecord with MySQLPlus

Late last week Rails core team announced the release of Rails 2.2RC1, which amongst other things slips in 'thread safety' and an implementation of a connection pool for our beloved ActiveRecord. Pratik Naik and Charles Nutter have both covered the implications of these contributions, but the executive summary is as follows: single ruby process (mongrel), plus many MySQL connections (ActiveRecord connection pool), means concurrent execution within that Mongrel process. This also means memory conservation, better throughput, and a much simplified deployment model. Sounds like something worth thinking about!

ActiveRecord Under The Hood

The ConnectionPool class introduced in Rails 2.2RC1 is effectively a simple wrapper for guaranteeing thread-safe access to a predefined set of connections (in our case, MySQL sessions). Let's take it for a spin outside of Rails:

> activerecord-pool.rb

require "rubygems"
require "active_record"
 
ActiveRecord::Base.establish_connection(
  :adapter => "mysql",
  :username => "root",
  :database => "database",
  :pool => 5
)
 
threads = []
10.times do |n|
 
 threads <<  Thread.new {
    ActiveRecord::Base.connection_pool.with_connection do |conn|
      res =  conn.execute("select sleep(1)")
    end
  }
end
 
# block and wait for all threads to finish
threads.each { |t| t.join }
 
# [root@localhost tm]# time ruby activerecord-pool-orig.rb
#
# real    0m10.644s
# user    0m0.405s
# sys     0m0.211s

Hold on, what happened? We've defined a pool with five MySQL connections, spooled up ten Ruby threads, and simulated some long running queries (one second each). How come it took 10 seconds to finish? It looks like it's executing our commands in serial!

To those of you familiar with the underlying gems this is hardly surprising. The problem is that the core MySQL gem uses blocking IO and every call to the MySQL server results in a completely locked up Ruby process - hence, no parallelism is possible. So much for our ConnectionPool?

Asynchronous MySQL with MySQLPlus

Thankfully, we have an alternative: MySQLPlus. Coming out of the Neverblock project, it attempts to solve exactly this problem by providing an 'asynchronous interface and threaded access support'. First, let's get it installed on our system:

git clone git://github.com/oldmoe/mysqlplus.git
cd mysqlpus
gem build mysqlplus.gemspec
gem install mysqlplus-0.1.0.gem

Now, let's see if we can patch ActiveRecord to behave as we want:

> activerecord-pool-mysqlplus.rb

require "rubygems"
require "active_record"
 
# Connection pool logic works, but the underlying driver
# is still blocking. However, aliasing the query method
# to use mysqlplus produces expected results!
require 'mysqlplus'
class Mysql; alias :query :async_query; end
 
ActiveRecord::Base.establish_connection(
  :adapter => "mysql",
  :username => "root",
  :database => "database",
  :pool => 5
)
 
threads = []
10.times do |n|
 
 threads <<  Thread.new {
    ActiveRecord::Base.connection_pool.with_connection do |conn|
      res =  conn.execute("select sleep(1)")
    end
  }
end
 
# block and wait for all threads to finish
threads.each { |t| t.join }
 
#[root@localhost tm]# time ruby activerecord-pool.rb
#
# real    0m2.663s
# user    0m0.413s
# sys     0m0.223s

rails-pool.zip (ActiveRecord source-code examples)

Downloads: 0 File Size: 1.7 KB

Much better! Instead of processing in serial order we were able to make use of our connection pool and complete the entire test in roughly two seconds. To achieve this, we've had to redefine the primary 'query' method in the MySQL gem to use the asynchronous interface provided by MySQLPlus - arguably not the cleanest way to handle the case. Having said that, MySQLPlus is a stable gem, and one you should watch as there are a number of interesting performance patches in the works.

Does this mean it work with Rails?

Almost. Sneaking MySQLPlus under the covers of ActiveRecord gets us pretty far, but Rails is not yet ready for production use with this pattern. First, Mongrel needs to be patched, and then the real bug hunt begins:

> rails-pool.rb

# simple controller to simulate a long running request
class SamplesController < ApplicationController
 def slow
  render :text => Sample.find_by_sql("select sleep(5)").to_s
 end
 
 def slow_pool
    ActiveRecord::Base.connection_pool.with_connection do |conn|
      res =  conn.execute("select sleep(5)")
    end
    render :text => 'done'
 end
end
 
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# append to config/environments/production.rb
config.threadsafe!
 
require 'mysqlplus'
class Mysql; alias :query :async_query; end
 
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# apache bench test, which issues two concurrent requests
 
#[root@localhost threadsafe]# ab -c 2 -n 2 localhost:3000/samples/slow
#
# Server Software:        Mongrel
# Server Hostname:        localhost
# Server Port:            3000
#
# Document Path:          /samples/slow
# Concurrency Level:      2
# Time taken for tests:   5.23008 seconds
# Complete requests:      2
 

As you can see, with a few simple changes our single Mongrel process is now capable of serving several parallel requests. However, more rigorous testing is still showing hung up threads and broken processing patterns (it is RC1 for a reason), but the bottom line is, we're making great progress!

Ruby Style Guides and Tools: How to Write Good Looking Ruby

Last week, Noel Rappin of Pathfinder Development wrote Elements of Ruby Style - an attempt at producing a Ruby "style guide." After some initial feedback to this, he's followed up with a response to some of the initial criticisms and suggestions.

Noel isn't the first to try to develop a Ruby style guide. In an attempt to promote Ruby's use within Google, Ian Macdonald wrote an extensive Ruby style guide back in 2006 (rather sadly, it appears his attempt did not work out).

What's a "Style Guide" then?

For those unfamiliar with style guides, the most famous for the English language are probably Strunk & White's The Elements of Style and The Chicago Manual of Style (my personal favorite) - both attempt to define how to write and lay out English texts properly. Style guides for programming languages, however, usually attempt to not only define the "correct" way to write and lay out code but also the right idioms to use in certain situations.

Other Tools

Ruby style and "code smells" have become topics of interest for many Ruby developers recently:

• Last month, Kevin Rutherford released reek, a "code smells detector"
• Marty Andrews released Roodi, a tool that checks Ruby code for complexity
• flog is also a popular Ruby tool for testing and showing the complexity of your Ruby code.

Supported by Media72 Hosting: Looking for reliable UK Ruby on Rails hosting? Packages start from just £7/month on our mod_rails/apache stack hosted on fast UK servers and include free 24/7 support.

Rails 2.2 RC1: i18n, thread safety, docs, etag/last-modified, JRuby/1.9 compatibility

Rails 2.2 is almost ready for its final release, but before we christen the gems, we’d like to have everyone test out a release candidate. Rails 2.2 is a major upgrade that includes a wealth of new features and fixes.

Chief inclusions are an internationalization framework, thread safety (including a connection pool for Active Record), easier access to HTTP caching with etags and last modified, compatibility with Ruby 1.9 and JRuby, and a wealth of new documentation.

Mike Gunderloy has compiled an exhaustive list and walk-through of many of the interesting new features for the Rails 2.2 release notes.

To help test the Rails 2.2 release candidate, please install with:
gem install rails -s http://gems.rubyonrails.org -v 2.2.0

Hopefully there will not be too much folly in the RC and we can quickly move to a final release. But it requires your help to get there.

Note that this release is called 2.2.0, not 2.1.99 as our previous naming scheme would have dictated. So the final release of Rails 2.2 will actually be 2.2.1 (if we only need one RC).

Merb 1.0 RC2 Now Available: Merb 1.0 Not Far Behind!

Following on from the highly successful MerbCamp, the Merb Framework took another big step this week with the announcement of the availability of release candidate 2 for it's upcoming 1.0 release (due within weeks now). No major changes and The Merbist blog lists mainly a focus on bug fixes and Windows compatibility:

The main focus for this release was to fix bugs and make the stack Windows compatible. We didn’t get any major bugs in RC1 but fixed a lot of small annoyances and problems with generated resources. We also made sure Merb itself would work properly with Windows (not using incompatible signals etc..) and we spent some time getting the Data Object sqlite3 drivers compiled on Windows.

Seems like a great time to begin playing around with Merb if you haven't yet. You can install / update Merb through RubyGems using sudo gem install merb

Ruby Inside is planning to run a resource packed feature on Merb when it hits 1.0, so keep your eyes peeled!

Support from: 1st Easy offers UK Rails hosting (dedicated and shared) running Phusion Passenger (mod_rails) and LAMP stack. If you want to get to know us first, or simply want to evaluate the performance of your Rails applications running on Passenger, we'll arrange a trial hosting account for you (full technical support included!)

Rails 2.1.2: Security, other fixes

Rails 2.1.2 includes the same two security fixes that we pushed out for 2.0.x recently. We’re talking about a backport of the offset/limit sanitization fix for Active Record and a fix against header-injection when using user-contributed strings in redirect_to (see Response Splitting for more information).

In addition, Rails 2.1.2 fixes the warning that users of RubyGems 1.3.0 were having with script/generate as well as a range of other minor fixes. Enjoy!

As always, you can install with:
gem install rails --version 2.1.2

Distributed Logging: Syslog-ng & Splunk

If you haven't already, I encourage you to take Splunk for a test drive. While there is no shortage of distributed log aggregators on the market, few if any come with as much flexibility and firepower when it comes to indexing and search when you need to find a needle in a haystack of log data.

After deploying it for a collection of Ruby applications using a simple UDP logger, I've decided to push the ball even further and see if I could route the entire stack into Splunk: HAproxy, Nginx, Ruby, and others, all over the network. After a few false starts (and not wanting to go to an Enterprise license), I've stumbled on a surprisingly easy solution: syslog-ng!

Routing Syslog-ng directly to Splunk

Syslog-ng was designed from the ground up to be a distributed logging application. Unlike Splunk, it is not meant to be a log analysis package, which is perfect for us, since that is exactly what we're using Splunk for in the first place. Let's take a look at a simple config file:

> syslog-ng.conf

# custom configuration for haproxy and Nginx
filter custom  { program("haproxy") or program("nginx"); };
 
log {
  # syslog
  source(s_sys);
 
  # filter everything but..
  filter(custom);
 
  # send to splunk, and local file
  destination(splunk);
  destination(local_log);
};
 
destination splunk { udp("192.168.0.198" port(9998)); };
destination local_log { file("/var/log/mylog.log"); };
 

syslog-ng.zip (Config files: syslog-ng, haproxy, nginx)

Downloads: 8 File Size: 25.5 KB

The configuration syntax for syslog-ng is a pleasure to work with: consume all messages from the syslog, apply a custom filter which we've defined to accept HAProxy and Nginx, and send the messages to a remote Splunk server and a local log file!

Routing HAProxy and Nginx to Syslog-ng

With syslog-ng ready to go, we have to update HAProxy and Nginx to start logging to syslog. The former is easy and requires a one line change (see top of haproxy config file), but the latter (Nginx) requires applying a patch to the source (no worries, it's stable - confirmed and in production). All done? Let's open up Splunk and issue a request that gets routed through HAProxy and to our Nginx server:

That's it, now you can see live logs from your Ruby, Haproxy, and Nginx processes stream live into your Splunk database for easy debugging and profiling. Of course, same procedures apply to any other process on a remote server - make it log to syslog, and you can route it to Splunk!

Mobile Orchard: Like Ruby Inside, but for iPhone Developers

A brief aside to mention my latest project (in collaboration with Dan Grigsby) - Mobile Orchard.

Mobile Orchard is a new iPhone developer news site, in a similar vein to Ruby Inside. We've already:

• filmed a 5 minute walkthrough of Android for iPhone developers
• shown 9 places you can go to promote your iPhone apps
• launched our podcast with a 30 minute interview with Hampton Catlin, well known in the Ruby and Rails scene (he invented Haml, for example) but who's also selling 1,000 copies a day of his iPhone application - we'll leave you to do the math at $1 per copy!
• and we're hoping to interview other Ruby and Rails developers using the iPhone platform! Could that be you? Get in touch!

There's a lot of money and a lot of excitement about the iPhone development community right now. Hampton Catlin says he's around the 48-50 mark in the "Top Paid iPhone App" list and he's selling 1,000 copies of his application each day, so if you can get into the top 10, you're going to be running a very serious business indeed.

Anyway, if you're an iPhone developer or are just interested in the platform, get over to Mobile Orchard and, ideally, subscribe to the feed or follow us on Twitter. If you don't want to do any of those things but want to wish us luck, you can do so in the comments here or save us on del.icio.us :)

How To Develop A Mac (Cocoa) Application With MacRuby And XCode

Over a year ago we had a post about how to build OS X GUI applications with Ruby and RubyCocoa. Since then, however, MacRuby has arrived on the scene. Not just the regular version of Ruby with some bindings to Cocoa, MacRuby is as native to OS X as JRuby is native to the JVM.

It's a significant development, then, that Apple has created a very in-depth tutorial called Developing Cocoa Applications Using MacRuby. It's very comprehensive. The tutorial introduces you to MacRuby, walks you through installation, demonstrates Objective-C bindings, and steps through using XCode and MacRuby to develop a simple GUI application. If you want to read a single tutorial and be able to develop a Mac GUI app with Ruby from start to finish, this is currently the tutorial to read.

That said, you might still appreciate our Ultimate List of RubyCocoa Tutorials, Tips, and Tools. There are 42 links there; many of which are still incredibly useful.

Rails Rumble 2008 applications

Want to have a look at the entries for this year’s Rails Rumble? Take a look at this list of applications that were submitted and then ordered according to category. Good stuff.

Paris on Rails: December 1st

Paris on Rails is having it’s third annual conference on December 1st. There’s a wealth of great speakers lined up and yours truly will be doing a video iChat session as well. If you register before November 9th, the entrance fee is just 80 euros. If you can, go!