*Yeah, yeah. Greek, Latin, who cares?

Wednesday, September 1, 2010

Recursion Pedagogy (or, How to Make Novice Programmers Crash their Machines)

It's fairly common to find programming books that teach recursion using the Fibonacci sequence as the main example. The obvious reasons for this are that the Fibonacci sequence is easy to understand (each value is just the sum of the preceding two values, with the first and second values in the sequence defined to both be 1) and that this simple dependence makes for massively simple recursion calls and tests. The recursive function, in fact, seems almost magical (code fragments usually in C# unless otherwise noted):

int RecursiveFibonacci(int nth)
{
    if (nth < 3)
    {
        return 1;
    }

    return RecursiveFibonacci(nth - 2) + RecursiveFibonacci(nth - 1);
}

That's it. That function will theoretically get you any number in the Fibonacci sequence (well, only up to the 32-bit limit, but that's easy enough to change).

Fibonacci numbers, however, are an incredibly bad candidate for recursion!

The function is short and simple, which is good; very easy to understand, which is better, and so massively inefficient that if you naively test it by asking for, say, the 50th Fibonacci number you're likely to spend some time trying to figure out where the bug is since it appears to have crashed!

What's actually happening, however, is that you're making an insane number of function calls, thus creating an insane number of function stack frames, each taking up at least eight bytes (if you're using 32-bit integers).

Calculating the 50th Fibonacci number by hand requires adding two numbers 48 times. Calculating the 50th Fibonacci number by recursion requires calling that function above 25.2 B-B-B-B-Billion times!! (Which requires more than 200 Gigabytes of memory just for the stack frames.) What's going on here? Well, we're looking at what I call the (w)Rec(k)Fib sequence:

1 1 3 5 9 15 25 41 67 109 177 287 465 753 1219 1973 ...

which looks very much like the Fibonacci sequence:

1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 ...

Every value in the (w)Rec(k)Fib sequence is twice its Fibonacci counterpart minus one. But that's not the most meaningful way to come up with it; rather, each value is the sum of the previous two (like Fibonacci) plus one more. So where Fibonacci is F(n) = F(n-1) + F(n-2), (w)Rec(k)Fib is w(n) = w(n-1) + w(n-2) + 1. It's also the number of times the recursive function is called to determine its Fibonacci sequence counterpart. When the function is called for, say, the fourth Fibonacci number, that's one call, to which we add the number of calls (3) for the third Fibonacci number and the number of calls (1) for the second Fibonacci number. In other words, we have to call the function far more times (twice minus one) than the value of the number we're seeking. Which means that even with modern processor speeds and memory sizes, and virtual memory, we can take an incredibly long time (or fail altogether). Want to calculate the hundredth Fibonacci number? (It's about 3.5E+20.) If you happen to have access to a computer that can manage a billion function calls per second (more actual operations, of course), it'll take around 22,500 years. But you can probably do it by hand in an hour or so (a couple if you're being careful not to miss a carry).

So, what's the right way to do it? Well, the simplest is through basic iteration, just like you'd do it by hand:

int IterativeFibonacciNumber(int nth)
{
    if (nth < 3)
    {
        return 1;
    }

    int a = 1;
    int b = 1;

    int temp;
    for (int n = 3; n <= nth; n++)
    {
       temp = b;
       b = b + a;
       a = temp;
    }

    return b;
}

It's not nearly as elegant. Nothing about it looks magical. It's not even all that easy to understand, though better variable names would obviously help. But it'll calculate the fiftieth Fibonacci number in about 4 milliseconds on my 2007-vintage laptop, running inside Visual Studio in Debug mode...which is where most of that 4 milliseconds comes from. How do I know? Switch most of the integer variables to long integers, and it'll calculate the 92nd Fibonacci number (the highest storable in a 64-bit integer) in the same 4 milliseconds.

Never, ever, ever use recursion if there's a straightforward way to accomplish the task without it. And, should you conclude that recursion is the way to go, try to at least get a feel for how the function behaves. If, as in this case, you're looking at O(c^n), then you almost certainly need to look again...or conclude that the problem is not practically soluble. (Unless the maximum n is small, of course.)

UPDATE: I should add, of course, that the quickest way is actually to open Excel, type 1 in the top left cell, type 1 again in the cell below it, then type =A1+A2 in the cell below that, and drag down on the box at the bottom-right corner of the cell. You'll get up to the 54th Fibonacci number with the full set of digits. Past that, you'll get floating point values with an ever-decreasing set of digits up to the 1476th Fibonacci number (1.307E+308). If you want the 'real' answer or values beyond that, you're going to have to set up your own kilobit-plus integer (or play with some identities that allow easier calculation of large Fibonacci numbers - see wikipedia).

11 comments:

  1. It's indeed insanely inefficient in terms of time, but not necessarily in terms of memory because at any point <= 50 of stack frames will reside on the stack at the same time.

    The technique that allows you to eliminate repeated calls to a recursive function with the same parameters is called memoization. Essentially it creates a look-up table, and before making a call you check the table for an answer. Assumes that there are no side effects :-)

    Elena

    ReplyDelete
  2. This is a topic which is near to my heart... Best wishes!
    Exactly where are your contact details though?
    My page ; income

    ReplyDelete
  3. I don't know whether it's just me or if everybody else experiencing problems with your website.

    It looks like some of the text on your posts
    are running off the screen. Can someone else please comment
    and let me know if this is happening to them as well? This could be a issue with
    my web browser because I've had this happen before. Many thanks
    Also see my webpage - high quality beats by dr. dre headphones

    ReplyDelete
  4. Whoa! This blog looks exactly like my old one! It's on a entirely different subject but it has pretty much the same layout and design. Great choice of colors!
    Feel free to visit my website ; Scott J. Ferrell

    ReplyDelete
  5. I savour, lead to I found just what I used to be taking a
    look for. You have ended my 4 day long hunt! God Bless you man.
    Have a nice day. Bye
    Here is my web blog :: Local News aggregation website

    ReplyDelete
  6. Does your site have a contact page? I'm having a tough time locating it but, I'd like to send you an email.
    I've got some suggestions for your blog you might be interested in hearing. Either way, great blog and I look forward to seeing it develop over time.
    My page ... iraqi dinar revalue

    ReplyDelete
  7. A person necessarily lend a hand to make significantly posts I
    might state. This is the very first time I frequented
    your web page and so far? I amazed with the analysis you made
    to create this particular post incredible. Excellent activity!
    Check out my web-site click the following website

    ReplyDelete
  8. I really love your blog.. Very nice colors & theme. Did you create this site yourself?

    Please reply back as I'm attempting to create my own blog and would like to find out where you got this from or what the theme is named. Appreciate it!

    Feel free to surf to my web site ... Summitt Energy Solutions

    ReplyDelete
  9. Hi, I do think this is an excellent blog. I stumbledupon it ;)
    I am going to come back once again since i have book-marked it.
    Money and freedom is the best way to change, may you be rich and continue to help other people.



    Feel free to visit my blog: dance

    ReplyDelete
  10. Why viewers still make use of to read news papers when in this technological world everything is existing on net?


    Here is my blog: Hot Naked Girls (http://xrl.us/bo9nw9)

    ReplyDelete
  11. Just desire to say yoiur article iis as surprising.
    The clarity in your post is simply great and i could think you are an expert on this subject.
    Fine with your permission let me to grasp
    your RSS feed tto keep up to date with approaching post.
    Thank you 1,000,000 and please keep up the gratifying work.


    my web siute brave frontier gem hack

    ReplyDelete