Better performance please

Over the last few months I've been pretty dissatisfied with the performance of this blog. Not only were page load times sometimes upwards of 10 seconds, but occasionally my swap usage would max out and crash the server, requiring a hard reboot. And it's a blog, for crying out loud—nothing this simple should ever flat-out crash a server, even if it's only got 256MB of RAM.