Midnight Oil

If you’ve been following our blog you may have noticed that we’re using Solr and ActsAsSolr plugin for our searching.

Getting started with acts_as_solr
acts_as_solr for development and production in one Tomcat instance
Optimizing Solr and Rails - Index in the background

Unfortunately being Java, Solr is a bit of a memory hog. We’ve attempted all sorts of optimizations, but we’re going to take a leap and switch to Sphinx. Sphinx is a free open-source SQL full-text search engine.

First step is to get Sphinx itself installed. For that have a look at Rob’s post:
http://www.notch8.com/articles/2007/10/15/sphinx-and-ultrasphinx-and-eye-on-search

Ultrasphinx works pretty much out of the box with will_paginate; however, a lot of our work is currently compatible with paginating_find. So to keep rails memory footprint down by using one plugin we’re sticking with paginating_find. Unfortunately Ultrasphinx doesn’t work out of the box with paginating_find, so time for a little hacking…

Install paginating_find plugin:

piston import http://svn.cardboardrocket.com/paginating_find/paginating_find vendor/plugins/paginating_find

More information on paginating_find

Install ultrasphinx plugin:

piston import http://fauna.rubyforge.org/svn/ultrasphinx/trunk/ vendor/plugins/ultrasphinx

More information on Ultrasphinx

Now for hacking Ultrasphinx plugin to work with Paginating Find plugin. Piston is great for managing plugins; however, it can easily get confused if you’ve hacked a plugin directly. The solution: use a method called “Evil Twin” as mentioned on the Err the Blog: Evil Twin Plugin

Create a directory for the hack:

mkdir vendor/plugins/ultrasphinx_hacks touch vendor/plugins/ultrasphinx_hacks/init.rb

Now edit the vendor/plugins/ultrasphinx_hacks/init.rb file with the hacks for paginating_find:

# Hack to allow Ultrasphinx to work with pagination_find Ultrasphinx::Search.module_eval do def first_page 1 end def last_page self.page_count end def previous_page? previous_page ? true : false end def next_page? next_page ? true : false end end

Now for adding a method to your controller for the search:

app/controllers/posts_controller.rb

def search @query = h(params[:query]) begin @posts = Ultrasphinx::Search.new(:query => @query, :class_names => 'Post', :page => params[:page] || 1, :per_page => 20) @ posts.run rescue RuntimeError flash[:warning] = 'Search is currently disabled. Please try again in a few hours.' rescue NoMethodError flash[:notice] = 'No records found for this search' end render :action => 'list' end

app/views/posts/list.html.erb

<ul> < % @posts.each do |cog| %> <li>< %= posts.name %></li> < % end %> </ul> < %= paginating_links(@posts) %>

Paginating End Result

Bonus : will_paginate styled links

Unfortunately paginating_find doesn’t automatically have the “style” that will_paginate automatically comes with. So to fix that just add a little css for the style and a partial for the pervious and next buttons.

.pagination{ padding: 2px; } .pagination a, .pagination a:visited{ padding: 0 5px; border: 1px solid #9aafe5; text-decoration: none; color: #2e6ab1; } .pagination a:hover, .pagination a:active{ border: 1px solid #2b66a5; color: #000; } .pagination .currentpage{ font-weight: bold; padding: 0 5px; border: 1px solid navy; background-color: #2e6ab1; color: #FFF; } .pagination .disablepage{ padding: 0 5px; border: 1px solid #929292; color: #929292; display: inline; }

And now for the partial:

app/views/shared/_paginate.html.erb

< % if (collection.is_a?(PagingEnumerator) || collection.is_a?(Ultrasphinx::Search)) && (collection.page_count != collection.first_page) -%> <div class="pagination"> < % if collection.previous_page? -%> < %= link_to '« previous', url_for(:overwrite_params => { :page => collection.previous_page }) %> < % else -%> <div class="disablepage">« previous</div> < % end -%> < %= paginating_links(collection, {:window_size => 4, :always_show_anchors => true, :params => {:query=> params[:query]}} ) %> < % if collection.next_page? -%> < %= link_to 'next »', url_for(:overwrite_params => { :page => collection.next_page }) %> < % else -%> <div class="disablepage">next »</div> < % end -%> </div> < % end -%>

So in your code replace the code paginating_links call with a render partial:

# OLD VERSION < %= paginating_links(@posts) %> # NEW VERSION < %= render :partial => 'shared/paginate', :locals => {:collection => @posts} %>

Styled Paginating End Result

Resources:
ErrTheBlog: Evil Twin Plugin
Will Paginate Plugin
Paginating Find Plugin
Ultrasphinx
Sphinx

We’ve taken a small leap and added several microformats to Obsidian Portal. It was just so easy, there was no reason not to. For starters, we’re going with hCard and XFN.

Right from the start, we’re going whole-hog on hCard. I made a helper function to output an hCard every time we display a user’s avatar. So, every comment, every recent update, every friend link has an hCard on it. If the microformat-aware spiders cannot figure out who is who on Obsidian Portal, then they’re brain-dead.

Secondly, we’ve added XFN rels on the friend-links in the user’s profile. I don’t know of any (non-trivial) tools that are using XFN yet, but it’s easy enough to add, so we did. We’ve got rel=me on links to user’s other websites, and even self-referential rel=me links on the profile pages. We’re basically screaming “I am me” at the spiders.

Still on the TODO list is rel-tag for the NPC character tags. We’ve got a lot of plans for tags in the future, we just need to free up some time to work.

In the future, I might look at adding hAtom for the Adventure Logs, but honestly, we’ve already got RSS feeds, and I can’t see the value-add here. Am I missing something? Let me know.

Examples

• My profile - hCard and XFN enabled
• The homepage - Notice the hCard for each recent-update user
• An Adventure Log post - Notice the hCards for the comments

Resources

• Microformats.org - The definitive source for information.
• Operator - A microformat parser plugin for Firefox. Adds a nice little toolbar that lights up when microformats are detected.
• Tails Export - Another microformat parser plugin for Firefox. Seems to handle de-duping better than Operator.
• rel-lint - A validator / checker for XFN and rel-tag

I got a new job! Starting April 23rd, I will be a Senior Software Engineer at Vitrue, helping them develop their embeddable video platform. I honestly don’t know a lot of details yet, but it seems to be a themeable, embeddable YouTube-like video platform. The twist is that companies can keep complete control over the posted content. From a corporate standpoint, that seems a little easier to stomach than the wild-wild-west of YouTube.

Best of all, I’ll be networking and meeting other young entrepreneurs like myself. The best place to meet startup-oriented people is, you guessed it, at a startup.

My greatest fear is that I will have less and less time to work on Obsidian Portal and RioFlexPay (link coming soon…). Oh well, I can always sleep less.

Before we switched to Slicehost, we had our VPS on Rimuhosting. I must say that I was very pleased with Rimuhosting’s service, and we never had any problems with stability. We eventually had to switch to Slicehost anyway because it was simply cheaper and we’re on razor-thin margins here.

Still, switching hosts is not a trivial process, and you have to factor in the switching effort when deciding if the price differential is worth it. If you’ve found a cheaper host and are thinking about jumping ship, the very first thing you should do is contact your current provider and see if they’ll match the new host’s price. That’s what we did initially with Rimuhosting. They didn’t match the deal, but they did give us a little more RAM for free. That tided us over for another month or two before we decided we wanted to give Slicehost a shot.

So, for posterity, and to help other bargain hunters, here is the letter I sent, and Rimuhosting’s response.

I’ve been a Rimuhosting customer for several months now. In general I am happy with the service, although my server seems to be getting more and more sluggish in recent weeks.

Due to the performance issues, I began shopping around for alternatives and I came across Slicehost ( http://www.slicehost.com/ ). Since they’re a competitor, I’m sure you’re already familiar with them.

According to their price list, they will give me 256MB RAM plus more disk space for $20/mo, compared to the 160MB of RAM I’m getting for $29. For me, the RAM is the big deal, and the extra storage is just icing on the cake.

I am seriously considering moving to slicehost, as I am finding that the 160MB I currently have is eaten quickly by Tomcat, mongrel, and MySQL. Plus, as I mentioned earlier, I have been seeing some very serious performance issues lately, with slowdowns on ssh as well as terrible response times from the web server. This could be due to not enough memory on my VPS, but I’m concerned it has to do with too few CPU cycles being devoted to the VPS.

In all honesty, I would rather not switch hosting companies, as I have been pleased with the level of service from Rimuhosting. In addition, moving a VPS installation is no small task and I do not relish the idea of trying to replicate all my settings on a new machine. Still, Slicehost’s pricing plan is very attractive.

Before I make any decisions, I would like to ask what sort of plan you would be willing to offer in order to compete with Slicehost. If you could offer a combination of more RAM and a lower price, it would go a long way towards tipping the scales in Rimuhosting’s favor.

Thanks for your time and I look forward to hearing from you.

Micah Wedemeyer
http://www.aisleten.com

Their response to me:

Hi Micah,

I’ve added some additional memory to your VPS:

[root@obsidianportal ~]# free -m
             total       used       free     shared    buffers     cached
Mem:           300        148        151          0         16         55
-/+ buffers/cache:         76        223
Swap:           95          0         95

Also moved your current pricing from 39.95, to where our current MiroVPS2 plans are, at 29.95 a month.

We feel that we offer a good value for the service we provide. We appreciate your business and hope you will stay with us.