I decided to remove the pagination and show the user the first set of records available. It then dawned on me that once you had multiple people working on the same page, they would all see the same records. This isn’t very efficient when you have a lot of stuff to get done. On a side note, this same problem existed in the paginated version, but when you have a link to move to the next page, it doesn’t crop up as much)

So now I needed to figure out a way to determine who was seeing what records, make sure that someone else didn’t see those same records at the same time, and make sure that if nothing was done to the records, they would return to the general pool in a reasonable amount of time.

Enter Redis.

I like redis. I’ve used it on a few other projects and felt that it would be a good fit since it has some nice set manipulation commands and the ability to expire a key after a specified amount of time. I started thinking about how to remove records after the user processed them and such, but I ran into some issues with possible AJAX search lag and that some records I didn’t really care about might be removed from the general population. Also, what do you do if the user only wanted to update a subset of those records? By modifying the key, I’d need to reset the expiration timer.

After writing a bit of code, I figured out that if I deleted the current user’s key before I make a union of all the users records, I wouldn’t have to worry about orphan records and unmodified records. Every time the user saw a list of records, only the current visible set would be removed from the general population.

Here’s a quick overview of what the script does.
2. Add the current user id to the set of curator ids
3. Remove the key that holds the current user’s reserved records
4. Get all the curator ids
5. Generate keys for all the curator reserved records
6. Get the union of all of the reserved records

I have to make a special note here. Redis expects arguments in the form SUNION “key1″ “key2″ “key3″. When you have an array in ruby, generally it will look something like ["key1", "key2", "key3"] The * (splat) operator splits out the array into a group of keys that works for the Redis command.

7-10. Find the first 15 currently available records and store the ids in the current user’s set in Redis.
11. Add a five minute expiration to the current user’s key.

It was interesting to me after thinking about it for a while that the solution ended up being so simple.

Anyway, this was probably blatantly obvious to a lot of people, but I figured I’d clear it up…

Email sent to Dreamhost back in August.

I’m going to make a request for this for you. Please note, it’s a significant bump from what we have right now, Rack 1.1.0, so only our rails expert can make the call, and check to make sure it won’t break anything. She’s on vacation this week, but I’ll send her a follow up email, so she can get back with you on this as soon as she’s back.

This was the response I got from Dreamhost after waiting for two months for an answer. (After I had to ask “hey, what’s up?”)

Thanks for writing. We have to be careful with the upgrade as we have a high number of dependency packages that need to go through the upgrade. It’s scheduled and being worked on, and the current estimate for the full upgrade to 3.0.1 is within a month of two. I’ll get back to you again when there is an update, but feel free to contact me if you have any more questions.

*Link includes my referral code

Looking at the parameter output I saw that the file was being sent to the server, but not processed. In my config.ru file I had use Rack::RawUpload, :paths => ['/searches/upload']to setup the middleware, but running “rake middleware” didn’t show it as being active. It seems there are two ways to activate the middleware and I was using the one that requires the server to be started with the config.ru file. The Cucumber/Capybara/Selenium uses Webrick which doesn’t start that way. So I moved my middleware initialization into initializers/middleware.rb and now things are working nicely.
Qtlhighliter::Application.config.middleware.use(Rack::RawUpload, :paths => ['/searches/upload'])

There is a bit of manual labor here which could be automated in some way you’d think, but for now it does work and that’s good enough for me.

1. Login as your deploy user.
2. Go to the deploy current directory where there is a Gemfile.
3. type ‘bundle config build.mysql –with-mysql-config=/usr/local/bin/mysql_config’

This will now append the commands to the end of the build command for mysql allowing it to build when you deploy your application. It works quite well and I think it’s a simple solution for one of the biggest issues I had with trying to use Bundler in production.

The ‘bundle config’ step also creates a .bundle/config in the user’s home directory that will then use those options for all bundle builds of that gem on the server. So if you’re installing 4 different apps on the server, you’ll only have to setup that command once and bundler will pick it up on each deploy.

Some examples
bundle config build.mysql --with-mysql-config=/usr/local/bin/mysql_config
bundle config build.rsruby --with-R-dir=/usr/local/lib/R/
bundle config build.rmagick --no-ri --no-rdoc


To view the options you can also type ‘bundle config’ alone in a directory that contains a Gemfile. (Not sure why it needs a Gemfile to exist, but again, not a hard requirement)

You have already activated rack 1.1.0, but your Gemfile requires rack 1.2.1. Consider using bundle exec.

So with the impending release of Rails 3 there is a little dependency that was added. Rack 1.2.1. Dreamhost is using passenger which also requires a version of Rack. The problem is that it’s 1.1.0. I’m pretty sure it’s not going to be possible to get Rails 3 running on Dreamhost without the servers being upgraded…

If you’re a Dreamhost customer please be sure to vote this suggestion up.
Get Ready for Rails 3

Please be sure to file a support ticket as well.

For me, if this isn’t fixed within the next few weeks I plan on moving to linode.com

You need to install logrotate since it doesn’t exist by default on the server and then place it in a location where it can be run. You also need to create the configuration file and status files. Once that is set up, you can install a cron job via the Dreamhost panel.

Now it should be rotating your application logs nightly. You can add more sites to the conf file as needed.

So I’ve been trying to get rails 3 running with bundler on Dreamhost for a while. I’ve had a few posts on here about how to do it. In the end, they didn’t work completely. I’ve sent requests into Dreamhost to upgrade rubygems to 1.3.6 so the newest versions of bundler would work. I was told ‘No’ but go to this wiki site.

I wasn’t too happy about that since the top of the page has a big warning ‘DON’T DO THIS’. If you’re here looking at this you probably have the tech knowhow to do this and realize the world won’t end when you do.

So here’s the step by step directions for how I got it working.
You can check out the original wiki instructions at http://wiki.dreamhost.com/RubyGems.

Add this to your .bashrc
This will setup your shell to use local gems installed in your .gems directory, setup the path to check there first and opt/bin as well. Next we need to install a newer version of rubygems.

This will get rubygems installed and make sure you run your version before dreamhost’s. You then install bundler and rake. These are the only two gems I install in the system as I prefer to have all of my apps use their own gem versions. Putting everything into system is a big mess and a dependency nightmare on updates.

Next we need to make sure your app is setup to use the bundler gem. You need to modify your environment.rb and boot.rb

Make sure you change the application name to your name. Really you’re only adding the single line for the GEM_PATH

I was using the suggestions in the wiki but I eventually figured out that Dreamhost wasn’t properly picking up my gems even with the changed GEM_PATH. Adding the ‘Gem.clear_paths’ to boot.rb allowed the gems to be seen. This is what finally cracked the problem.

Hopefully this works for you.