Steve Souders of Google is speaking on Even Faster Web Sites. I've read Steve's book and loved it. It's the kind of book you read in the morning, use to make changes to your site in the afternoon and at the end of the day, you've made a huge difference.
Usually, a small percentage of the time (10%) a browser spends putting a page in front of the user is spent downloading the HTML document. Making the Web server faster might save compute time or storage, but it doesn't do much for the user's perceived response time.
80-90% of the end user response time is spent on the front end of the page load experience, so start there. You'll have a greater potential for impact. The changes are simpler than backend tuning. And finally, they've been proven to work. I can personally vouch for that.
Steve created rules or high performance Web sites and built them into a Firefox extension called YSlow. Running in on www.windley.com, shows that I don't do so well. I get a grade of "F." I'll have to work on that!
Long inline scripts block rendering and download. You can initiate execution with a setTimeout, move it to an external script, or use the defer attribute (IE only). Don't scatter them in the page and don't put it between the stylesheet and any other resource.