About this site

Subscribe to the RSS Feed

Making search possible on a static site

Update 3/8/2012 Feedparser docs are actually available online again!

One of the challenges I identified early one when setting up this site was that it was going to be a bit tricky to get some kind of search tool working for it. When I Googled it, I was pretty excited to find this tool but when I sat down yesterday to implement it, it was broke. And by broke, I mean it wasn’t returning any results and the developers weren’t responding to any attempts at communications. So, I figured, if they can parse my RSS feed and turn it into something that returns search results, so can I. What follows is how I managed to get that going.

AppEngine, again It seems like recently I’ve found quite a few uses for AppEngine. And this is without even really having it on my radar before last week. Besides serving this site from it, a coworker and I put together a little email handler for a Wufoo form on AppEngine earlier this week. So, when my dream of making this site only rely on external web services for interactivity fell apart at some point yesterday, I guess it seemed like a natural fit.

Really, all that the search widget over there in the right sidebar does is do a jQuery $.getJson call to the AppEngine app which grabs my RSS feed from Feedburner, parses it, looks for the query string in the content of the feed, and then returns a JSONP reponse. Here’s what that looks like on the AppEngine side:

Since the client is calling a Cross Domain resource here, the response needs to be a JSONP response in order for jQuery to be cool with it (hence wrapping the json in the callback there). The other really, really cool thing I found was Feedparser. The project is very mature but, unfortunately, the docs were on a domain that was owned by Mark Pilgrim (he was a heavy contributor to the project, apparently) so when he performed his recent info-suicide, that domain went with him. They’re putting docs up onto PyPi soon (at least that’s what the chatter in their forums is saying)

Anyways, it’s pretty easy to use. Import the module, give it a URL (or a sting with XML in it) and whammo, you’ve got a Python dictionary with all the stuff from the feed in it. Since the docs are temporarily gone, I can’t really say what other types of feeds or XML formats it accepts but it worked for the one that I’m generating from WordPress.

One other thing about that code that you’ll notice is that instead of doing the normal from google.appengine.ext import webapp I’m doing import webapp2 as webapp and I’m also able to do import json even though AppEngine is still on Python 2.5. This is because in my app.yaml, I told it to use Python 2.7:

On the client-side, I’m just taking the response and sticking it over in a div underneath the search form:

So, I guess that’s pretty much it. That’s how you get get search working on a static site. It would be really cool to see these guys get their act together but, I guess it was kinda fun to do it on my own as well.

blog comments powered by Disqus