The comics aggregator is now significantly faster for items that load from RSS feeds, specifically:
- Ask a Manager
- Basic Instructions
- Carolyn Hax
- Damon Young (yes, I know he hasn’t published anything yet this year; I hold out hope for his swift return!)
- Dry Bones
- Miss Manners
- This Modern World
- xkcd.com
This is especially true for the items in bold above, because they all comes from The Washington Post, whose RSS feeds are particularly slow to load. For those items you should be seeing a speedup of 10-20 seconds per item loading the page.
I can now say with confidence that the page should nearly always load in less than a second, so if that’s not what you’re seeing, you should let me know so I can investigate and fix it.
For those who are curious about backend technical details, here are the two issues which I tracked down and fixed today to achieve this speed-up:
- The script was trying to cache the feeds so as not to have to request them every time, but the caching layer was refusing by default to cache responses with undefined content lengths. I told it to start caching them.
- I was periodically scanning all these feeds in the background, so that the fetching would happen and be cached then rather than it needing to happen in real time when someone asked the aggregator to display one of these items. However, the way I was doing the caching, the feeds were being cached in the wrong place, so the server couldn’t see the cached data when handling page load requests from users.
