properly rendering 404 errors from inside a rails application

I just migrated a site that had a bunch of links that have been in in the search engines for a while. Oddly it seems that the only thing hitting those links seem to be the crawlers themselves. I needed a way to invalidate those links, since I couldn’t create a proper redirect because of changing IDs.

/records/show/12345 used to be valid, but has been replaced with the RESTful version /records/00123. The ID is now also meaningful instead of a MySQL generated id.

My first attempt was to just redirect to the 404 page.
record = Record.find(params[:id]
rescue ActiveRecord::RecordNotFound
redirect_to("/404.html")

But as I watched the logs, I noticed that this really wasn’t right since it was still returning a 302 (redirect) and the a 200 (OK) code for those links. The crawlers were getting the instruction that you should just display the 404 page for those links. That might seem OK, but really I wanted them to get the 404 immediately and remove the page from their databases.

record = Record.find(params[:id]
rescue ActiveRecord::RecordNotFound
render(:file => "#{RAILS_ROOT}/public/404.html", :layout => false, :status => 404)

By rendering the 404.html directly and including the 404 status code, it should help to fix the situation.

Map/Reduce real world examples

Why is it every example of Map/Reduce is counting words in a big textfile? Isn’t there someplace offering something of substance?

Also, the examples also seem to gloss over the fact that you need to get the data to the nodes in order to process it. Google’s got GFS hooked up to all 1000+ nodes, but how do you pull that off on EC2? EBS is only hooked to one node at a time…

I’m working on a system to do large scale analyses, and hopefully will be posting updates on the issues I’ve found.

You can follow the project at http://github.com/mcwbbc/vipdac/tree/master