I recently finished up a bit of a weekend hack project which involved wrangling quite a lot of quite a lot of data into something that could be easily viewed on the web. In case you ended up here first, you can check that out over here. Basically, I took the 5,000,000 or so publicly available crime reports in Chicago since 2001 and compared it to the observed temperature data for that same time period to see if it was true that, as the temperature in Chicago goes up, so does the crime rate. Short answer: yes. More nuanced answer: mostly. What follows is an attempt to break down how I did what I did from a slightly more technical perspective in case anyone else wants to give this kind of thing a shot. SPOILER ALERT: It’s actually pretty easy.

So, this morning I finished what I started working on when I had a few spare moments at this year’s PyCon. Namely, take this site, convert it to use something that is an actual static site generator (preserving all the functions that it had before) and host it on S3.

I’ve written about my very limited exploits in exploring and leveraging 2010 census data in the past but this weekend I finally got back around to doing something about it. What I ended up with is looking like a pretty useful tool for anyone looking at projecting census data onto arbitrary geographies.

My introduction to Python, and Django in particular, was a week-long project to setup, from scratch, a basic, working GeoDjango project. Thanks to some really stellar documentation, which has actually improved quite a bit since back in the day (like, a year and a half ago or whatever), I was able to do it. But, quite frankly, that was only the beginning. When I showed off my little application around the office, it ended up being a full fledged project, which I’m still tweaking to this day. This is an attempt to unravel some of what I’ve picked up during that time, especially given all of the recent developments in the web based mapping sphere. This is by no means a comprehensive overview, but rather just a guy sharing how he does what he does.

One of the challenges I identified early one when setting up this site was that it was going to be a bit tricky to get some kind of search tool working for it. When I Googled it, I was pretty excited to find this tool but when I sat down yesterday to implement it, it was broke. And by broke, I mean it wasn’t returning any results and the developers weren’t responding to any attempts at communications. So, I figured, if they can parse my RSS feed and turn it into something that returns search results, so can I. What follows is how I managed to get that going.

After attending this meetup last week, I was pretty excited to get going on hacking census data for my own purposes. I had the day off on Thursday so I spent most of the day trying to get the census.ire.org source code running in a local VM. Despite being one of those guys that’s pretty good at following directions, I did run into a few problems (but, then again, I’m not much of a sysadmin). In the spirit of maybe making someone’s life easier, here’s how I got the project going in a Ubuntu 11.04 vagrant VM.

Part of what I do at work is make maps. To me, the best maps are those that show you, at a glance, something interesting about what’s going on around you, and, if you’re curious, allow you to explore what’s going on in the rest of the world. I’ve been following what the Tribapps team has been doing with the 2010 census data as it has been coming out and after going to a recent Hacks/Hackers Meetup where Joe Germuska presented some of the APIs that they’ve been working on and I gotta say, I was inspired. Below are some preliminary steps and observations.

Back in January, I read this and thought it was a pretty cool idea. For some reason, I just kinda filed it away and didn’t really think about it until sometime in the early morning hours of October 22 when I got the urge to write about this but didn’t really have anywhere to do it. Add to that the fact that I my brain starts hurting when I start to see the long road of configuring and deploying a site, whatever the platform. I just wanted something that would kinda be there, look presentable out of the box, and not have anything to do with PHP.