Building XMLs with the magic of method_missing in Ruby

"Any sufficiently advanced technology is indistinguishable from magic."
- Arthur C Clarke

"Any sufficiently advanced technology which you don't understand is magic."
- Reddit comments

Method missing is an elegant dynamic programming trick. The best use of it is in the dynamic finders in ActiveRecord. Another simple but awesome library which uses the same trick is the XML builder library. This blogpost illustrates the use of method missing by building a rudimentary XML builder library. I haven't checked out XML builder source code to keep it simple and authentic.

The following snippet illustrates usage of this builder. To keep it simple i'm skipping XML attributes, comments and DTDs:

require 'buildr'
xml = Buildr.new
puts xml.phonebook {
xml.contact {
xml.full_name 'John Doe'
xml.email 'john.doe@gmail.com'
xml.phone '121-101'
}
xml.contact {
xml.full_name 'William Smith'
xml.email 'william.smith@gmail.com'
xml.phone '121-102'
}
}


And this is the XML output expected from the builder:

<phonebook>
<contact>
<full_name>John Doe</full_name>
<email>john.doe@gmail.com</email>
<phone>121-101</phone>
</contact>
<contact>
<full_name>William Smith</full_name>
<email>william.smith@gmail.com</email>
<phone>121-102</phone>
</contact>
</phonebook>


Looking at the usage, its obvious that the Buildr uses method missing to interpret missing methods as valid tags. Another pattern is the usage of blocks for nesting tags. Let's get started with a Builder class which implements method_missing to dynamically render XML tags:

class Buildr
def method_missing(tag,*args,&block)
content = args.empty? ? yield : args.first
render(tag,content)
end

def render(tag, content)
buffer = ""
buffer += "<#{tag}>"
buffer += content
buffer += "</ #{tag}>"
end
end


render method creates opening and closing tags and puts text or further evaluation of nested tags between them. That's the essence of what we need in the succinct implementation above. Let's run it:

<phonebook><contact><phone>121-102</phone></contact></phonebook>


Boink! That's predictable for a first cut. But here's what went wrong. yield returns the value of the last statement in the block, but what we need is an aggregate of all the xml outputs in a block. That's why the output contains only the last phone number of the last Contact.

The fix was elusive, but what we need here is some way to aggregate the outputs of each method_missing call and return that as the output of the block. I fixed it by adding a buffer (instance_variable) to aggregate xml outputs in a block and resetting the buffer for each block.

class Buildr
def initialize
@buffer = ""
end

def method_missing(tag,*args,&block)
render(tag) do
unless args.empty?
args.first
else
@buffer = ""
output = yield
output
end
end
end

def render(tag, &content)
@buffer += "<#{tag}>"
@buffer += yield
@buffer += "</ #{tag}>"
end
end


render method now takes a block, which returns text or evaluates nested blocks. The block also takes care of resetting buffer. Now let's run it:

<phonebook>
<contact><full_name>John Doe</full_name><email>john.doe@gmail.com</email><phone>121-101</phone></contact>
<contact><full_name>William Smith</full_name><email>william.smith@gmail.com</email><phone>121-102</phone></contact>
</phonebook>


That's awesome. It works, but its not formatted and uses instance variable state which could have been avoided.

PS: This experimental Buildr is hosted at Github

Aspect oriented blocks

Programmers being lazy want to avoid redundant code. An example is the boilerplate code written over and over again whenever you access a file or a database connection: Opening the file, Reading/Writing from the filestream and then do some house-keeping work to make sure the file is properly closed.

Look at the following example of writing a file in Java:


BufferedWriter out;
try{
out = new BufferedWriter(new FileWriter("out.file"));
out.write("stuff");

}catch(IOException e) {
logger.error("Error opening file: " + e);

}finally{
out.close();
}


This is why i never feel comfortable writing a one-off file program in Java. Sure, you can abstract this in a function called writeToFile(filename,contents) and re-use it. But every java programmer in the world has to write this atleast once. If i were to write a standard library API, i would never want my users to suffer.

This is a hard problem to abstract especially because you want to do something before writing to a file, do some stuff after writing. This is called around advice (before+after) in Aspect-oriented programming. If you have looked at the usability of AOP in Java, you'd rather repeat code. This is where blocks come to the rescue in Ruby. Look at the same example in Ruby.

File.open('out.file','w') do |f|
f.write 'stuff'
end

The problem of opening, closing and reading/writing to stream has been abstracted once and for all, and as a programmer you just have to care about reading/writing. This is an elegant solution to the same problem.

Now how can you apply the same technique in your day-to-day Ruby programming. Let's say you're writing a cool desktop app in Ruby and it works in all platforms - Windows, Linux and MacOSX. Assume you're storing user preferences in different directories in different platforms and you want to unit test this behaviour. Let's say everytime you test a platform, you change PLATFORM constant, test the behavior and then reset it to it's original value.

describe('user preferences') do

before do
@app.start
end

it 'should be stored in MyDocuments in Windows' do
original_platform = PLATFORM
PLATFORM = 'Windows'

@app.pref_file.location.should == "C:\\MyDocuments\\myapp.preferences"

PLATFORM = original_platform
end

it 'should be stored in ~/.myapp in Linux' do
original_platform = PLATFORM
PLATFORM = 'Linux'

@app.pref_file.location.should == '~/.myapp'

PLATFORM = original_platform
end

it 'should be stored in Users/john/Preferences/myapp.plist in MacOSX' do
original_platform = PLATFORM
PLATFORM = 'MacOSX'

@app.pref_file.location.should == '/Users/john/Preferences/myapp.plist'

PLATFORM = original_platform
end

end

That's a lot of boilerplate code to switch PLATFORM, not to mention the numerous warnings you get in reassigning CONSTANTs. This can be elegantly solved using blocks.

The blocks provide little sandboxes in which your test code can execute with PLATFORM set to a specific value. Once you come out of the block, PLATFORM is reset back to it's original value.


describe('user preferences') do
before do
@app.start
end

it 'should be stored in MyDocuments in Windows' do
os('Windows') do
@app.pref_file.location.should == "C:\\MyDocuments\\myapp.preferences"
end
end

it 'should be stored in ~/.myapp in Linux' do
os('Linux') do
@app.pref_file.location.should == '~/.myapp'
end
end

it 'should be stored in Users/john/Preferences/myapp.plist in MacOSX' do
os('MacOSX') do
@app.pref_file.location.should == '/Users/john/Preferences/myapp.plist'
end
end

def os(platform, &block)
original_platform = PLATFORM
PLATFORM = platform
yield
PLATFORM = original_platform
end

end

Having the boilerplate code in one place, you can refactor it to remove and reassign Constants to eliminate warnings:

def os(platform, &block)
original_platform = Object.send(:remove_const, 'PLATFORM')
Object.const_set('PLATFORM', platform)
yield
Object.const_set('PLATFORM', original_platform)
end


Tim Toady

Tim Toady / TIMTOWTDI is a programming motto from Perl users. It stands for There Is More Than One Way To Do It. Ruby is inspired by Perl and also follows the same principle.

For example, consider the problem of looping. It may be hard to believe but there are 9 different ways to loop in Ruby.



100.times do
puts "I will not throw paper airplanes in class"
end

1.upto(100) do |i|
puts "#{i}. I will not throw paper airplanes in class"
end

for i in 1..100
puts "#{i}. I will not throw paper airplanes in class"
end


Look at the different solutions for the famous 99 bottles of beer.



i = 99
while i >= 0 do
bottles = "#{i.zero? ? 'No more' : i} bottles"
bottles.chop! if i == 1
puts "Take one down and pass it around, #{bottles} of beer on the wall.\n" unless i == 99
puts "#{bottles} of beer on the wall, #{bottles} of beer."
i -= 1
end
puts "Go to the store and buy some more, 99 bottles of beer on the wall."

i = 99
until i < 0 do
bottles = "#{i.zero? ? 'No more' : i} bottles"
bottles.chop! if i == 1
puts "Take one down and pass it around, #{bottles} of beer on the wall.\n" unless i == 99
puts "#{bottles} of beer on the wall, #{bottles} of beer."
i -= 1
end
puts "Go to the store and buy some more, 99 bottles of beer on the wall."

i = 99
loop do
bottles = "#{i.zero? ? 'No more' : i} bottles"
bottles.chop! if i == 1
puts "Take one down and pass it around, #{bottles} of beer on the wall.\n" unless i == 99
puts "#{bottles} of beer on the wall, #{bottles} of beer."
i -= 1
break if i < 0
end
puts "Go to the store and buy some more, 99 bottles of beer on the wall."


There are several choices for even looping through the elements 
of an array.Consider dealing a deck of cards in a game. 



suits = ['♥','♠','♦','♣']
cards = 2..10.to_a + ['J','Q','K','A']
deck = suits.collect {|suit| cards.collect {|card| "#{card}#{suit}"}}.flatten.shuffle

deck.each do |card|
  deal card
end

for card in deck
  deal card
end

deck.each_with_index do |card,i|
  deal "Player#{i%4+1}", card
end

The number of choices gives the programmer the freedom to choose a looping construct based on the expressiveness for a particular problem.

Currying Curry functions in Javascript

Currying is a technique of transforming a function that takes multiple arguments into a partial application.

Here's an Example from Wikipedia:

f(x,y) = y / x.

g(y) = f(2,y) = y / 2

g(3) = 3 / 2.

At each partial application we end up with an executable function.

While reading about Currying in Functional programming for the rest of us, i came across this example:


pow(x,y) = x**y
square(x) = pow(x,2) = x**2


The idea of calling a function with partial arguments and getting back another function seemed interesting. I set out to implement it in, well Javascript, which is the only functional programming language i have ever used.


function pow(x,y){
return y == 0 ? 1 : x * pow(x,y-1);
}


square can be implemented in an imperative way by reusing pow:


function square(x){
return pow(x,2)
}


or by currying/partial application of pow:


var square = c(pow,2);
var cube = c(pow,3)
var sixteen = c(pow,2,4);


c is a function which allows currying, by remembering the curried args and returning a partial function which uses those arguments when calling the original function.


function c(f){
var cargs = Array.prototype.slice.call(arguments,1);
return function(){
Array.prototype.push.apply(arguments, cargs)
return f.apply(this, arguments);
}
}


The curried functions can be called like regular functions:


console.log(square(5));
console.log(sixteen())


Now if you notice, it works only if you want to partially apply args at the end. Consider another example, where you might want to apply curried args in the beginning:


function greet(message,name){
return message + ' ' + name;
}

function hello(name){
return greet('hello',name);
}

function goodbye(name){
return greet('goodbye',name);
}


Curried implementation of hello and goodbye would look like


var hello = c(greet,'hello');
var goodbye = c(greet,'goodbye');


c fn can only apply args at the end, so let's create another curry function which applies arguments at the beginning:


//curry args at the beginning
function _c(f){
var cargs = Array.prototype.slice.call(arguments,1);
return function(){
Array.prototype.push.apply(cargs, arguments)
return f.apply(this, cargs);
}
}

//curry args at the end
function c_(f){
var cargs = Array.prototype.slice.call(arguments,1);
return function(){
Array.prototype.push.apply(arguments, cargs)
return f.apply(this, arguments);
}
}

var square = c_(pow,2);
var cube = c_(pow,3)
var hello = _c(say,'hello');
var goodbye = _c(say,'goodbye');


c_ and _c is rather odd to denote the order of currying. Other than order, there is relatively no difference between the two functions. So let's refactor the curry functions with a higher order function:


function c(order){
return function(f){
var cargs = Array.prototype.slice.call(arguments,1);
return function(){
return f.apply(this, order(arguments,cargs));
}
}
}


c_ and _c can be obtained by currying c, a higher order function which takes order function as input and returns a curry function.


var _c = c(currybefore);
var c_ = c(curryafter);


currybefore and curryafter are functions which decide the order of currying:


function currybefore(args,curry_args){
return concat(curry_args,args);
}

function curryafter(args,curry_args){
return concat(args,curry_args);
}

function concat(list1,list2){
var result = [];
Array.prototype.push.apply(result,list1);
Array.prototype.push.apply(result,list2);
return result;
}


If you're tired after currying curry functions, go have some butter chicken curry:

---

* Array.prototype.push.apply(arguments, args) is a way to do pushall with push in javascript bcos apply splats array args
* Array.prototype.slice.call(arguments,1) is an idiom in javascript since arguments is not an actual array

Google's theory of Recursion

Counting with Fingers

Primitive arithmetic operations +, -, *, /, ** are taken for granted in most programming languages. But a language need not necessarily support primitive arithmetic operations for it to be turing complete. Increment and decrement are the basic operations available in certain minimal turing complete languages like BF. How would you go about performing addition, multiplication, exponential in such a language with just Increment and decrement operations? It's certainly possible, but a short digression on the history of counting devices follows.

Counting was one of the primary reasons the computer was invented. There were numerous counting devices before the computer. The most primitive way was the tally marks found in egyptian cave drawings. Counting with fingers was also a common method as taught in primary schools. Chinese invented the abacus to aid in counting. Then came the mechanical and electronic calculators and computers.











How will you implement addition without +, multiplication without *, and exponentiation without ** ? Exponentiation is just repeated multiplication. Multiplication is just repeated addition and addition is just....well, repeated incrementation.

Think of the way you were taught addition in school. How did you add 2 and 3? You kept 2 fingers in one hand(left), 3 in the other (right).







You deduct one from left and add one to right.







You deduct one from left and add one to right.







Voila! You have no more fingers left in your left hand and the answer in your right hand. And all you needed to do was increment and decrement in alternate hands.

The only limitation is you have only five fingers in each hand. Even with the five fingers in each leg, it isn't much. Counting in this way isn't practical for larger number unless you have infinite fingers in each hand. But computers have a large memory (analogical fingers). And it is possible to add numbers using just increment and decrement as demonstrated previously.

An implementation of this in Scheme would look like this. [Scheme has primitive add1 and sub1 for increment and decrement]


(define (add n x)
(cond
[(zero? n) x]
[else (add1 (add (sub1 n) x))]
)
)


This is what happens if we use this function to add 5 and 8


(add 5 8)
= (1 + (add 4 8))
= (1 + (1 + (add 3 8)))
= (1 + (1 + (1 + (add 2 8))))
= (1 + (1 + (1 + (1 + (add 1 8)))))
= (1 + (1 + (1 + (1 + (1 + (add 0 8))))))
= (1 + (1 + (1 + (1 + (1 + 8)))))
= (1 + (1 + (1 + (1 + 9))))
= (1 + (1 + (1 + 10)))
= (1 + (1 + 11))
= (1 + 12)
= (13)


We have successfully used increment recursively to implement addition. Addition is a primitive recursive function. The same applies for Subtraction as well.

Now to implement multiplication without *, you can implement it recursively using + or add that we implemented just now:

(define (multiply n x)
(cond
[(zero? n) 0]
[else (add x (multiply (sub1 n) x))]
)
)

This is what happens if we use this function to multiply 4 and 5


(multiply 4 5)
= (5 + (multiply 3 5))
= (5 + (5 + (multiply 2 5)))
= (5 + (5 + (5 + (multiply 1 5))))
= (5 + (5 + (5 + (5 + (multiply 0 5)))))
= (5 + (5 + (5 + (5 + 0))))


...Before we reduce it further, we have a choice. We can abstract the addition as understood previously or delve deeper into how the addition is performed recursively.

= (5 + (5 + (5 + 5)))

= (5 + (5 + 10))

OR

(5 + (5 + (1 + (1 + (1 + (1 + (1 + 5)))))))

finally evaluating to

= 20

Now to implement exponent without **, you can implement it recursively using * or multiply that we implemented just now, which in turn is recursively implemented using + or add:


(define (exponent x n)
(cond
[(zero? n) 1]
[else (multiply x (exponent x (sub1 n)))]
)
)


This is what happens if we use this function to calculate 2**3


(exponent 2 3)
= (2 * (exponent 2 2))
= (2 * (2 * (exponent 2 1)))
= (2 * (2 * (2 * (exponent 2 0))))
= (2 * (2 * (2 * 1)))


The multiplications are evaluated recursively through addition, and the additions are evaluated recursively through increment.

Arithmetic operations can be computed recursively was a re-revelation to me on reading 'how to design programs' long after forgetting how to count with fingers.

Brainf****d

I got brainf****d on a Saturday afternoon. No i didn't get admitted for psychological reasons. Brainf*** is an esoteric programming language. Its a turing complete language with as few as 8 instructions:

>	Increment the pointer
<	Decrement the pointer
+	Increment the byte at the pointer
-	Decrement the byte at the pointer
.	Output the byte at the pointer
,	Input a byte and store it in the byte at the pointer
[	Jump forward past the matching ] if the byte at the pointer is zero
]	Jump backward to the matching [ unless the byte at the pointer is zero

Click here for a brief description of the language. The wikipedia entry is more detailed spoiling the fun in discovering the quirks of the language.

I started writing an interpreter for BF in Ruby, which was a simple switch. But the interesting part is actually writing code in BF. Its much more challenging than Assembly language because of the instruction set and using a tape like memory without any registers.

Here is the BF interpreter in Ruby:

if ARGV.length != 1
 puts "Usage: ruby brainfuck.rb filename"
 exit
end
mem = Array.new(30000,0x00)
loopstack = Array.new
p = 0

code  = File.open(ARGV[0]).readlines.join("")

i=0
while i < code.length
 case code[i]
 when '>'
   p += 1
 when '<'
   p -= 1
 when '+'
   mem[p] += 1
 when '-'
   mem[p] -= 1
 when '.'
   print mem[p].chr
 when ','
   mem[p] = STDIN.getc
 when '['
   if mem[p].zero?
     i = code.index(']',i)
   else
     loopstack.push i
   end
 when ']'
   if(mem[p].zero?)
     loopstack.pop
   else
     i = loopstack.last
   end
 end
 i+=1
end

And here is the traditional HelloWorld program in BF:

#H
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++
.
>
#e
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
+
.
>
#l
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++
.
>
#l
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++
.
>
#o
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++
.
>
#SPACE
++++++++++++++++++++++++++++++++
.
>
#w
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++
.
>
#o
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++
.
>
#r
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++
.
>
#l
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++
.
>
#d
++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++
.

So long for HelloWorld? This would probably be the worst language for beginners!

The only work in the Interpreter was managing loops expressed using [ and ]. I ended up using a stack to manage loops similar to the way stacks are used in procedural languages to transfer control to methods and return back.

Here is a program to iterate from 0 to 9 in BF:

#ASCII value of 0
++++++++++++++++++++++++++++++++++++++++++++++++
>
#Loop counter variable
++++++++++
[
< . + >
-
]